Jul 21 12:37:45.081732 ip-10-0-130-2 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 21 12:37:45.081747 ip-10-0-130-2 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 21 12:37:45.081801 ip-10-0-130-2 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 21 12:37:45.082268 ip-10-0-130-2 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 21 12:37:55.314261 ip-10-0-130-2 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 21 12:37:55.314280 ip-10-0-130-2 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 1c03b5aa1ee04827b38576d5edc90fcc --
Jul 21 12:39:46.586322 ip-10-0-130-2 systemd[1]: Starting Kubernetes Kubelet...
Jul 21 12:39:46.965953 ip-10-0-130-2 kubenswrapper[2556]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 21 12:39:46.965953 ip-10-0-130-2 kubenswrapper[2556]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 21 12:39:46.965953 ip-10-0-130-2 kubenswrapper[2556]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 21 12:39:46.965953 ip-10-0-130-2 kubenswrapper[2556]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 21 12:39:46.965953 ip-10-0-130-2 kubenswrapper[2556]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 21 12:39:46.967057 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.966938    2556 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 21 12:39:46.971953 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.971938    2556 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 21 12:39:46.971953 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.971954    2556 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 21 12:39:46.972029 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.971958    2556 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 21 12:39:46.972029 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.971961    2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 21 12:39:46.972029 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.971964    2556 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 21 12:39:46.972029 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.971967    2556 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 21 12:39:46.972029 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.971971    2556 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 21 12:39:46.972029 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.971974    2556 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 21 12:39:46.972029 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.971978    2556 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 21 12:39:46.972029 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.971982    2556 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 21 12:39:46.972029 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.971986    2556 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 21 12:39:46.972029 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.971989    2556 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 21 12:39:46.972029 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.971992    2556 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 21 12:39:46.972029 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.971994    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 21 12:39:46.972029 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.971997    2556 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 21 12:39:46.972029 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.972000    2556 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 21 12:39:46.972029 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.972002    2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 21 12:39:46.972029 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.972005    2556 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 21 12:39:46.972029 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.972007    2556 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 21 12:39:46.972029 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.972022    2556 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 21 12:39:46.972029 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.972026    2556 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 21 12:39:46.972485 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.972029    2556 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 21 12:39:46.972485 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.972031    2556 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 21 12:39:46.972485 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.972034    2556 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 21 12:39:46.972485 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.972037    2556 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 21 12:39:46.972485 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.972040    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 21 12:39:46.972485 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.972043    2556 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 21 12:39:46.972485 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.972045    2556 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 21 12:39:46.972485 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.972048    2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 21 12:39:46.972485 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.972050    2556 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 21 12:39:46.972485 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.972053    2556 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 21 12:39:46.972485 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.972056    2556 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 21 12:39:46.972485 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.972058    2556 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 21 12:39:46.972485 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.972060    2556 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 21 12:39:46.972485 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.972064    2556 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 21 12:39:46.972485 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.972067    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 21 12:39:46.972485 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.972070    2556 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 21 12:39:46.972485 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.972073    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 21 12:39:46.972485 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.972076    2556 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 21 12:39:46.972485 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.972079    2556 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 21 12:39:46.972967 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.972082    2556 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 21 12:39:46.972967 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.972084    2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 21 12:39:46.972967 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.972087    2556 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 21 12:39:46.972967 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.972089    2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 21 12:39:46.972967 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.972091    2556 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 21 12:39:46.972967 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.972095    2556 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 21 12:39:46.972967 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.972097    2556 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 21 12:39:46.972967 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.972100    2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 21 12:39:46.972967 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.972102    2556 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 21 12:39:46.972967 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.972105    2556 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 21 12:39:46.972967 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.972108    2556 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 21 12:39:46.972967 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.972112    2556 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 21 12:39:46.972967 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.972115    2556 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 21 12:39:46.972967 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.972118    2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 21 12:39:46.972967 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.972121    2556 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 21 12:39:46.972967 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.972124    2556 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 21 12:39:46.972967 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.972126    2556 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 21 12:39:46.972967 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.972129    2556 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 21 12:39:46.972967 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.972132    2556 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 21 12:39:46.973457 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.972134    2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 21 12:39:46.973457 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.972137    2556 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 21 12:39:46.973457 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.972139    2556 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 21 12:39:46.973457 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.972142    2556 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 21 12:39:46.973457 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.972144    2556 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 21 12:39:46.973457 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.972147    2556 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 21 12:39:46.973457 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.972149    2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 21 12:39:46.973457 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.972152    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 21 12:39:46.973457 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.972155    2556 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 21 12:39:46.973457 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.972157    2556 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 21 12:39:46.973457 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.972160    2556 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 21 12:39:46.973457 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.972162    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 21 12:39:46.973457 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.972165    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 21 12:39:46.973457 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.972167    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 21 12:39:46.973457 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.972170    2556 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 21 12:39:46.973457 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.972172    2556 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 21 12:39:46.973457 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.972175    2556 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 21 12:39:46.973457 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.972177    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 21 12:39:46.973457 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.972181    2556 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 21 12:39:46.973457 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.972183    2556 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 21 12:39:46.973937 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.972186    2556 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 21 12:39:46.973937 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.972188    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 21 12:39:46.973937 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.972191    2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 21 12:39:46.973937 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.972193    2556 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 21 12:39:46.973937 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.972196    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 21 12:39:46.973937 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.972198    2556 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 21 12:39:46.973937 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.972200    2556 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 21 12:39:46.973937 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.972203    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 21 12:39:46.973937 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.972207    2556 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 21 12:39:46.973937 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.972209    2556 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 21 12:39:46.973937 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.972212    2556 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 21 12:39:46.973937 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.972214    2556 feature_gate.go:328] unrecognized feature gate: Example
Jul 21 12:39:46.973937 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.972217    2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 21 12:39:46.973937 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.972220    2556 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 21 12:39:46.973937 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.972223    2556 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 21 12:39:46.973937 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.972225    2556 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 21 12:39:46.973937 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.972227    2556 feature_gate.go:328] unrecognized feature gate: Example2
Jul 21 12:39:46.973937 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.972230    2556 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 21 12:39:46.973937 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.972232    2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 21 12:39:46.973937 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.972235    2556 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 21 12:39:46.974425 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.973487    2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 21 12:39:46.974425 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.973495    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 21 12:39:46.974425 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.973498    2556 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 21 12:39:46.974425 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.973501    2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 21 12:39:46.974425 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.973504    2556 feature_gate.go:328] unrecognized feature gate: Example2
Jul 21 12:39:46.974425 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.973513    2556 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 21 12:39:46.974425 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.973516    2556 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 21 12:39:46.974425 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.973519    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 21 12:39:46.974425 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.973522    2556 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 21 12:39:46.974425 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.973525    2556 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 21 12:39:46.974425 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.973527    2556 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 21 12:39:46.974425 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.973530    2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 21 12:39:46.974425 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.973532    2556 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 21 12:39:46.974425 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.973535    2556 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 21 12:39:46.974425 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.973538    2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 21 12:39:46.974425 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.973542    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 21 12:39:46.974425 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.973545    2556 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 21 12:39:46.974425 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.973547    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 21 12:39:46.974425 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.973550    2556 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 21 12:39:46.974425 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.973552    2556 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 21 12:39:46.974942 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.973555    2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 21 12:39:46.974942 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.973557    2556 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 21 12:39:46.974942 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.973560    2556 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 21 12:39:46.974942 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.973562    2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 21 12:39:46.974942 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.973565    2556 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 21 12:39:46.974942 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.973567    2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 21 12:39:46.974942 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.973570    2556 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 21 12:39:46.974942 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.973572    2556 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 21 12:39:46.974942 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.973574    2556 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 21 12:39:46.974942 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.973577    2556 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 21 12:39:46.974942 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.973580    2556 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 21 12:39:46.974942 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.973582    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 21 12:39:46.974942 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.973585    2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 21 12:39:46.974942 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.973587    2556 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 21 12:39:46.974942 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.973589    2556 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 21 12:39:46.974942 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.973592    2556 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 21 12:39:46.974942 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.973594    2556 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 21 12:39:46.974942 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.973597    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 21 12:39:46.974942 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.973605    2556 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 21 12:39:46.975430 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.973608    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 21 12:39:46.975430 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.973611    2556 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 21 12:39:46.975430 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.973613    2556 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 21 12:39:46.975430 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.973615    2556 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 21 12:39:46.975430 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.973618    2556 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 21 12:39:46.975430 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.973620    2556 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 21 12:39:46.975430 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.973623    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 21 12:39:46.975430 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.973633    2556 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 21 12:39:46.975430 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.973635    2556 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 21 12:39:46.975430 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.973638    2556 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 21 12:39:46.975430 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.973641    2556 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 21 12:39:46.975430 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.973643    2556 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 21 12:39:46.975430 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.973646    2556 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 21 12:39:46.975430 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.973648    2556 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 21 12:39:46.975430 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.973652    2556 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 21 12:39:46.975430 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.973656    2556 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 21 12:39:46.975430 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.973659    2556 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 21 12:39:46.975430 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.973662    2556 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 21 12:39:46.975430 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.973664    2556 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 21 12:39:46.975913 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.973667    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 21 12:39:46.975913 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.973670    2556 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 21 12:39:46.975913 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.973672    2556 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 21 12:39:46.975913 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.973675    2556 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 21 12:39:46.975913 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.973677    2556 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 21 12:39:46.975913 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.973680    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 21 12:39:46.975913 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.973683    2556 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 21 12:39:46.975913 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.973686    2556 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 21 12:39:46.975913 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.973689    2556 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 21 12:39:46.975913 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.973692    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 21 12:39:46.975913 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.973699    2556 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 21 12:39:46.975913 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.973701    2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 21 12:39:46.975913 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.973704    2556 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 21 12:39:46.975913 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.973707    2556 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 21 12:39:46.975913 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.973710    2556 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 21 12:39:46.975913 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.973713    2556 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 21 12:39:46.975913 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.973715    2556 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 21 12:39:46.975913 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.973718    2556 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 21 12:39:46.975913 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.973720    2556 feature_gate.go:328] unrecognized feature gate: Example
Jul 21 12:39:46.975913 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.973723    2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 21 12:39:46.976414 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.973725    2556 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 21 12:39:46.976414 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.973728    2556 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 21 12:39:46.976414 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.973731    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 21 12:39:46.976414 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.973733    2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 21 12:39:46.976414 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.973736    2556 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 21 12:39:46.976414 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.973738    2556 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 21 12:39:46.976414 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.973740    2556 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 21 12:39:46.976414 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.973743    2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 21 12:39:46.976414 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.973745    2556 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 21 12:39:46.976414 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.973748    2556 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 21 12:39:46.976414 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.973765    2556 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 21 12:39:46.976414 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.973771    2556 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 21 12:39:46.976414 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.973775    2556 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 21 12:39:46.976414 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.973778    2556 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 21 12:39:46.976414 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.973781    2556 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 21 12:39:46.976414 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.973784    2556 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 21 12:39:46.976414 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.973787    2556 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 21 12:39:46.976414 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.973790    2556 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 21 12:39:46.976414 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.973792    2556 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 21 12:39:46.976414 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.973796    2556 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 21 12:39:46.976909 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.973800    2556 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 21 12:39:46.976909 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.974191    2556 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 21 12:39:46.976909 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.974197    2556 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 21 12:39:46.976909 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.974202    2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 21 12:39:46.976909 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.974207    2556 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 21 12:39:46.976909 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.974210    2556 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 21 12:39:46.976909 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.974213    2556 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 21 12:39:46.976909 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.974216    2556 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 21 12:39:46.976909 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.974219    2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 21 12:39:46.976909 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.974221    2556 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 21 12:39:46.976909 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.974224    2556 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 21 12:39:46.976909 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.974226    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 21 12:39:46.976909 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.974229    2556 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 21 12:39:46.976909 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.974232    2556 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 21 12:39:46.976909 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.974234    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 21 12:39:46.976909 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.974237    2556 feature_gate.go:328] unrecognized feature gate: Example
Jul 21 12:39:46.976909 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.974239    2556 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 21 12:39:46.976909 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.974242    2556 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 21 12:39:46.976909 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.974244    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 21 12:39:46.976909 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.974246    2556 feature_gate.go:328] unrecognized feature gate: Example2
Jul 21 12:39:46.977501 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.974249    2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 21 12:39:46.977501 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.974251    2556 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 21 12:39:46.977501 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.974254    2556 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 21 12:39:46.977501 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.974257    2556 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 21 12:39:46.977501 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.974259    2556 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 21 12:39:46.977501 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.974262    2556 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 21 12:39:46.977501 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.974264    2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 21 12:39:46.977501 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.974268    2556 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 21 12:39:46.977501 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.974270    2556 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 21 12:39:46.977501 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.974273    2556 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 21 12:39:46.977501 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.974275    2556 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 21 12:39:46.977501 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.974277    2556 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 21 12:39:46.977501 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.974280    2556 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 21 12:39:46.977501 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.974282    2556 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 21 12:39:46.977501 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.974284    2556 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 21 12:39:46.977501 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.974287    2556 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 21 12:39:46.977501 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.974290    2556 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 21 12:39:46.977501 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.974292    2556 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 21 12:39:46.977501 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.974297    2556 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 21 12:39:46.977501 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.974300    2556 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 21 12:39:46.977974 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.974303    2556 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 21 12:39:46.977974 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.974306    2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 21 12:39:46.977974 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.974308    2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 21 12:39:46.977974 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.974311    2556 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 21 12:39:46.977974 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.974313    2556 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 21 12:39:46.977974 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.974316    2556 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 21 12:39:46.977974 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.974319    2556 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 21 12:39:46.977974 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.974321    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 21 12:39:46.977974 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.974324    2556 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 21 12:39:46.977974 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.974326    2556 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 21 12:39:46.977974 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.974329    2556 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 21 12:39:46.977974 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.974332    2556 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 21 12:39:46.977974 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.974335    2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 21 12:39:46.977974 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.974337    2556 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 21 12:39:46.977974 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.974340    2556 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 21 12:39:46.977974 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.974342    2556 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 21 12:39:46.977974 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.974345    2556 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 21 12:39:46.977974 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.974349    2556 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 21 12:39:46.977974 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.974351    2556 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 21 12:39:46.977974 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.974354    2556 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 21 12:39:46.978481 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.974356    2556 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 21 12:39:46.978481 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.974358    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 21 12:39:46.978481 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.974361    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 21 12:39:46.978481 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.974363    2556 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 21 12:39:46.978481 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.974365    2556 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 21 12:39:46.978481 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.974368    2556 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 21 12:39:46.978481 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.974370    2556 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 21 12:39:46.978481 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.974372    2556 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 21 12:39:46.978481 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.974375    2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 21 12:39:46.978481 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.974377    2556 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 21 12:39:46.978481 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.974380    2556 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 21 12:39:46.978481 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.974382    2556 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 21 12:39:46.978481 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.974385    2556 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 21 12:39:46.978481 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.974387    2556 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 21 12:39:46.978481 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.974389    2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 21 12:39:46.978481 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.974392    2556 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 21 12:39:46.978481 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.974394    2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 21 12:39:46.978481 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.974399    2556 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 21 12:39:46.978481 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.974402    2556 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 21 12:39:46.978937 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.974404    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 21 12:39:46.978937 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.974406    2556 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 21 12:39:46.978937 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.974409    2556 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 21 12:39:46.978937 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.974411    2556 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 21 12:39:46.978937 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.974414    2556 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 21 12:39:46.978937 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.974416    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 21 12:39:46.978937 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.974418    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 21 12:39:46.978937 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.974421    2556 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 21 12:39:46.978937 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.974423    2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 21 12:39:46.978937 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.974426    2556 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 21 12:39:46.978937 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.974428    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 21 12:39:46.978937 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.974431    2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 21 12:39:46.978937 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.974433    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 21 12:39:46.978937 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.974436    2556 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 21 12:39:46.978937 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.974438    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 21 12:39:46.978937 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.974441    2556 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 21 12:39:46.978937 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.974443    2556 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 21 12:39:46.978937 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.974446    2556 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 21 12:39:46.978937 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.974449    2556 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 21 12:39:46.978937 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.974452    2556 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 21 12:39:46.979444 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.974455    2556 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 21 12:39:46.979444 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.974527    2556 flags.go:64] FLAG: --address="0.0.0.0"
Jul 21 12:39:46.979444 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.974534    2556 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 21 12:39:46.979444 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.974541    2556 flags.go:64] FLAG: --anonymous-auth="true"
Jul 21 12:39:46.979444 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.974545    2556 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 21 12:39:46.979444 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.974549    2556 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 21 12:39:46.979444 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.974553    2556 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 21 12:39:46.979444 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.974557    2556 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 21 12:39:46.979444 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.974561    2556 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 21 12:39:46.979444 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.974564    2556 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 21 12:39:46.979444 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.974568    2556 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 21 12:39:46.979444 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.974572    2556 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 21 12:39:46.979444 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.974575    2556 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 21 12:39:46.979444 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.974578    2556 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 21 12:39:46.979444 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.974581    2556 flags.go:64] FLAG: --cgroup-root=""
Jul 21 12:39:46.979444 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.974586    2556 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 21 12:39:46.979444 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.974588    2556 flags.go:64] FLAG: --client-ca-file=""
Jul 21 12:39:46.979444 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.974591    2556 flags.go:64] FLAG: --cloud-provider="external"
Jul 21 12:39:46.979444 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.974594    2556 flags.go:64] FLAG: --cluster-dns="[]"
Jul 21 12:39:46.979444 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.974598    2556 flags.go:64] FLAG: --cluster-domain=""
Jul 21 12:39:46.979444 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.974601    2556 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 21 12:39:46.979444 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.974604    2556 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 21 12:39:46.979444 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.974607    2556 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 21 12:39:46.979444 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.974610    2556 flags.go:64] FLAG: --container-log-max-files="5"
Jul 21 12:39:46.980005 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.974613    2556 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 21 12:39:46.980005 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.974616    2556 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 21 12:39:46.980005 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.974619    2556 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 21 12:39:46.980005 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.974622    2556 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 21 12:39:46.980005 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.974625    2556 flags.go:64] FLAG: --contention-profiling="false"
Jul 21 12:39:46.980005 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.974627    2556 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 21 12:39:46.980005 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.974630    2556 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 21 12:39:46.980005 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.974633    2556 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 21 12:39:46.980005 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.974637    2556 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 21 12:39:46.980005 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.974643    2556 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 21 12:39:46.980005 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.974646    2556 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 21 12:39:46.980005 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.974649    2556 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 21 12:39:46.980005 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.974651    2556 flags.go:64] FLAG: --enable-load-reader="false"
Jul 21 12:39:46.980005 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.974654    2556 flags.go:64] FLAG: --enable-server="true"
Jul 21 12:39:46.980005 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.974657    2556 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 21 12:39:46.980005 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.974662    2556 flags.go:64] FLAG: --event-burst="100"
Jul 21 12:39:46.980005 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.974665    2556 flags.go:64] FLAG: --event-qps="50"
Jul 21 12:39:46.980005 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.974668    2556 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 21 12:39:46.980005 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.974674    2556 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 21 12:39:46.980005 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.974677    2556 flags.go:64] FLAG: --eviction-hard=""
Jul 21 12:39:46.980005 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.974680    2556 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 21 12:39:46.980005 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.974683    2556 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 21 12:39:46.980005 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.974686    2556 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 21 12:39:46.980005 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.974690    2556 flags.go:64] FLAG: --eviction-soft=""
Jul 21 12:39:46.980005 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.974693    2556 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 21 12:39:46.980605 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.974696    2556 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 21 12:39:46.980605 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.974699    2556 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 21 12:39:46.980605 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.974702    2556 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 21 12:39:46.980605 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.974705    2556 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 21 12:39:46.980605 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.974707    2556 flags.go:64] FLAG: --fail-swap-on="true"
Jul 21 12:39:46.980605 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.974710    2556 flags.go:64] FLAG: --feature-gates=""
Jul 21 12:39:46.980605 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.974714    2556 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 21 12:39:46.980605 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.974717    2556 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 21 12:39:46.980605 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.974720    2556 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 21 12:39:46.980605 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.974723    2556 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 21 12:39:46.980605 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.974726    2556 flags.go:64] FLAG: --healthz-port="10248"
Jul 21 12:39:46.980605 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.974729    2556 flags.go:64] FLAG: --help="false"
Jul 21 12:39:46.980605 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.974732    2556 flags.go:64] FLAG: --hostname-override="ip-10-0-130-2.ec2.internal"
Jul 21 12:39:46.980605 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.974735    2556 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 21 12:39:46.980605 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.974737    2556 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 21 12:39:46.980605 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.974740    2556 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 21 12:39:46.980605 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.974744    2556 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 21 12:39:46.980605 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.974747    2556 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 21 12:39:46.980605 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.974750    2556 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 21 12:39:46.980605 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.974753    2556 flags.go:64] FLAG: --image-service-endpoint=""
Jul 21 12:39:46.980605 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.974756    2556 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 21 12:39:46.980605 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.974758    2556 flags.go:64] FLAG: --kube-api-burst="100"
Jul 21 12:39:46.980605 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.974762    2556 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 21 12:39:46.981238 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.974765    2556 flags.go:64] FLAG: --kube-api-qps="50"
Jul 21 12:39:46.981238 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.974768    2556 flags.go:64] FLAG: --kube-reserved=""
Jul 21 12:39:46.981238 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.974772    2556 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 21 12:39:46.981238 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.974774    2556 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 21 12:39:46.981238 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.974778    2556 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 21 12:39:46.981238 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.974780    2556 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 21 12:39:46.981238 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.974783    2556 flags.go:64] FLAG: --lock-file=""
Jul 21 12:39:46.981238 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.974787    2556 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 21 12:39:46.981238 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.974790    2556 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 21 12:39:46.981238 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.974792    2556 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 21 12:39:46.981238 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.974797    2556 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 21 12:39:46.981238 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.974800    2556 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 21 12:39:46.981238 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.974803    2556 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 21 12:39:46.981238 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.974806    2556 flags.go:64] FLAG: --logging-format="text"
Jul 21 12:39:46.981238 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.974808    2556 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 21 12:39:46.981238 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.974812    2556 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 21 12:39:46.981238 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.974814    2556 flags.go:64] FLAG: --manifest-url=""
Jul 21 12:39:46.981238 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.974817    2556 flags.go:64] FLAG: --manifest-url-header=""
Jul 21 12:39:46.981238 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.974821    2556 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 21 12:39:46.981238 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.974824    2556 flags.go:64] FLAG: --max-open-files="1000000"
Jul 21 12:39:46.981238 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.974828    2556 flags.go:64] FLAG: --max-pods="110"
Jul 21 12:39:46.981238 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.974830    2556 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 21 12:39:46.981238 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.974833    2556 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 21 12:39:46.981238 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.974836    2556 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 21 12:39:46.981238 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.974839    2556 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 21 12:39:46.981856 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.974841    2556 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 21 12:39:46.981856 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.974844    2556 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 21 12:39:46.981856 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.974848    2556 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 21 12:39:46.981856 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.974854    2556 flags.go:64] FLAG: --node-status-max-images="50"
Jul 21 12:39:46.981856 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.974857    2556 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 21 12:39:46.981856 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.974860    2556 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 21 12:39:46.981856 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.974863    2556 flags.go:64] FLAG: --pod-cidr=""
Jul 21 12:39:46.981856 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.974867    2556 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0"
Jul 21 12:39:46.981856 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.974872    2556 flags.go:64] FLAG: --pod-manifest-path=""
Jul 21 12:39:46.981856 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.974875    2556 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 21 12:39:46.981856 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.974879    2556 flags.go:64] FLAG: --pods-per-core="0"
Jul 21 12:39:46.981856 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.974882    2556 flags.go:64] FLAG: --port="10250"
Jul 21 12:39:46.981856 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.974885    2556 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 21 12:39:46.981856 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.974888    2556 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-02f79c946f39b978a"
Jul 21 12:39:46.981856 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.974892    2556 flags.go:64] FLAG: --qos-reserved=""
Jul 21 12:39:46.981856 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.974895    2556 flags.go:64] FLAG: --read-only-port="10255"
Jul 21 12:39:46.981856 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.974898    2556 flags.go:64] FLAG: --register-node="true"
Jul 21 12:39:46.981856 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.974901    2556 flags.go:64] FLAG: --register-with-taints=""
Jul 21 12:39:46.981856 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.974904    2556 flags.go:64] FLAG: --registry-burst="10"
Jul 21 12:39:46.981856 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.974907    2556 flags.go:64] FLAG: --registry-qps="5"
Jul 21 12:39:46.981856 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.974910    2556 flags.go:64] FLAG: --reserved-cpus=""
Jul 21 12:39:46.981856 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.974912    2556 flags.go:64] FLAG: --reserved-memory=""
Jul 21 12:39:46.981856 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.974916    2556 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 21 12:39:46.981856 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.974919    2556 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 21 12:39:46.981856 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.974922    2556 flags.go:64] FLAG: --rotate-certificates="false"
Jul 21 12:39:46.982473 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.974925    2556 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 21 12:39:46.982473 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.974928    2556 flags.go:64] FLAG: --runonce="false"
Jul 21 12:39:46.982473 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.974930    2556 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 21 12:39:46.982473 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.974933    2556 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 21 12:39:46.982473 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.974936    2556 flags.go:64] FLAG: --seccomp-default="false"
Jul 21 12:39:46.982473 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.974939    2556 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 21 12:39:46.982473 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.974942    2556 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 21 12:39:46.982473 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.974945    2556 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 21 12:39:46.982473 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.974948    2556 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 21 12:39:46.982473 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.974951    2556 flags.go:64] FLAG: --storage-driver-password="root"
Jul 21 12:39:46.982473 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.974954    2556 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 21 12:39:46.982473 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.974957    2556 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 21 12:39:46.982473 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.974959    2556 flags.go:64] FLAG: --storage-driver-user="root"
Jul 21 12:39:46.982473 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.974962    2556 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 21 12:39:46.982473 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.974965    2556 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 21 12:39:46.982473 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.974968    2556 flags.go:64] FLAG: --system-cgroups=""
Jul 21 12:39:46.982473 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.974971    2556 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 21 12:39:46.982473 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.974977    2556 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 21 12:39:46.982473 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.974980    2556 flags.go:64] FLAG: --tls-cert-file=""
Jul 21 12:39:46.982473 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.974983    2556 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 21 12:39:46.982473 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.974987    2556 flags.go:64] FLAG: --tls-min-version=""
Jul 21 12:39:46.982473 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.974993    2556 flags.go:64] FLAG: --tls-private-key-file=""
Jul 21 12:39:46.982473 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.974996    2556 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 21 12:39:46.982473 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.974999    2556 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 21 12:39:46.982473 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.975002    2556 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 21 12:39:46.983214 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.975004    2556 flags.go:64] FLAG: --v="2"
Jul 21 12:39:46.983214 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.975008    2556 flags.go:64] FLAG: --version="false"
Jul 21 12:39:46.983214 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.975027    2556 flags.go:64] FLAG: --vmodule=""
Jul 21 12:39:46.983214 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.975031    2556 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 21 12:39:46.983214 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.975034    2556 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 21 12:39:46.983214 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.975152    2556 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 21 12:39:46.983214 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.975155    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 21 12:39:46.983214 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.975159    2556 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 21 12:39:46.983214 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.975162    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 21 12:39:46.983214 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.975165    2556 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 21 12:39:46.983214 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.975167    2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 21 12:39:46.983214 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.975170    2556 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 21 12:39:46.983214 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.975173    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 21 12:39:46.983214 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.975175    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 21 12:39:46.983214 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.975178    2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 21 12:39:46.983214 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.975181    2556 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 21 12:39:46.983214 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.975183    2556 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 21 12:39:46.983214 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.975186    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 21 12:39:46.983214 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.975189    2556 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 21 12:39:46.983214 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.975191    2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 21 12:39:46.983214 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.975194    2556 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 21 12:39:46.984065 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.975196    2556 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 21 12:39:46.984065 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.975199    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 21 12:39:46.984065 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.975209    2556 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 21 12:39:46.984065 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.975213    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 21 12:39:46.984065 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.975215    2556 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 21 12:39:46.984065 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.975218    2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 21 12:39:46.984065 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.975220    2556 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 21 12:39:46.984065 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.975224    2556 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 21 12:39:46.984065 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.975227    2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 21 12:39:46.984065 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.975229    2556 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 21 12:39:46.984065 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.975232    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 21 12:39:46.984065 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.975234    2556 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 21 12:39:46.984065 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.975236    2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 21 12:39:46.984065 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.975239    2556 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 21 12:39:46.984065 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.975241    2556 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 21 12:39:46.984065 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.975244    2556 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 21 12:39:46.984065 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.975246    2556 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 21 12:39:46.984065 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.975249    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 21 12:39:46.984065 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.975251    2556 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 21 12:39:46.984065 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.975254    2556 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 21 12:39:46.984625 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.975256    2556 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 21 12:39:46.984625 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.975258    2556 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 21 12:39:46.984625 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.975261    2556 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 21 12:39:46.984625 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.975263    2556 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 21 12:39:46.984625 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.975266    2556 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 21 12:39:46.984625 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.975268    2556 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 21 12:39:46.984625 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.975271    2556 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 21 12:39:46.984625 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.975273    2556 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 21 12:39:46.984625 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.975275    2556 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 21 12:39:46.984625 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.975278    2556 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 21 12:39:46.984625 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.975280    2556 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 21 12:39:46.984625 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.975283    2556 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 21 12:39:46.984625 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.975285    2556 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 21 12:39:46.984625 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.975288    2556 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 21 12:39:46.984625 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.975290    2556 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 21 12:39:46.984625 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.975294    2556 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 21 12:39:46.984625 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.975297    2556 feature_gate.go:328] unrecognized feature gate: Example2
Jul 21 12:39:46.984625 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.975300    2556 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 21 12:39:46.984625 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.975303    2556 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 21 12:39:46.984625 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.975307    2556 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 21 12:39:46.985164 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.975310    2556 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 21 12:39:46.985164 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.975312    2556 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 21 12:39:46.985164 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.975314    2556 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 21 12:39:46.985164 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.975317    2556 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 21 12:39:46.985164 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.975319    2556 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 21 12:39:46.985164 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.975322    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 21 12:39:46.985164 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.975325    2556 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 21 12:39:46.985164 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.975327    2556 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 21 12:39:46.985164 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.975330    2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 21 12:39:46.985164 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.975332    2556 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 21 12:39:46.985164 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.975335    2556 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 21 12:39:46.985164 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.975337    2556 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 21 12:39:46.985164 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.975340    2556 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 21 12:39:46.985164 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.975343    2556 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 21 12:39:46.985164 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.975346    2556 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 21 12:39:46.985164 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.975349    2556 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 21 12:39:46.985164 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.975352    2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 21 12:39:46.985164 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.975355    2556 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 21 12:39:46.985164 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.975358    2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 21 12:39:46.985638 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.975360    2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 21 12:39:46.985638 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.975363    2556 feature_gate.go:328] unrecognized feature gate: Example
Jul 21 12:39:46.985638 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.975366    2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 21 12:39:46.985638 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.975368    2556 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 21 12:39:46.985638 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.975371    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 21 12:39:46.985638 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.975373    2556 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 21 12:39:46.985638 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.975376    2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 21 12:39:46.985638 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.975379    2556 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 21 12:39:46.985638 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.975382    2556 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 21 12:39:46.985638 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.975385    2556 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 21 12:39:46.985638 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.975388    2556 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 21 12:39:46.985638 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.975390    2556 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 21 12:39:46.985638 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.975394    2556 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 21 12:39:46.985638 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.975396    2556 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 21 12:39:46.985638 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.975400    2556 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 21 12:39:46.985638 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.975403    2556 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 21 12:39:46.985638 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.975407    2556 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 21 12:39:46.985638 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.975410    2556 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 21 12:39:46.985638 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.975413    2556 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 21 12:39:46.986113 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.975416    2556 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 21 12:39:46.986113 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.975419    2556 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 21 12:39:46.986113 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.975422    2556 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 21 12:39:46.986113 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.975424    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 21 12:39:46.986113 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.975427    2556 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 21 12:39:46.986113 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.976046    2556 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 21 12:39:46.986113 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.984839    2556 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 21 12:39:46.986113 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.984854    2556 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 21 12:39:46.986113 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.984874    2556 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 21 12:39:46.986113 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.984880    2556 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 21 12:39:46.986113 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.984945    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 21 12:39:46.986113 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.984950    2556 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 21 12:39:46.986113 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.984954    2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 21 12:39:46.986113 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.984956    2556 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 21 12:39:46.986113 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.984960    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 21 12:39:46.986488 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.984962    2556 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 21 12:39:46.986488 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.984965    2556 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 21 12:39:46.986488 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.984968    2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 21 12:39:46.986488 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.984970    2556 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 21 12:39:46.986488 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.984972    2556 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 21 12:39:46.986488 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.984975    2556 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 21 12:39:46.986488 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.984978    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 21 12:39:46.986488 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.984980    2556 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 21 12:39:46.986488 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.984983    2556 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 21 12:39:46.986488 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.984986    2556 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 21 12:39:46.986488 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.984989    2556 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 21 12:39:46.986488 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.984991    2556 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 21 12:39:46.986488 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.984994    2556 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 21 12:39:46.986488 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.984998    2556 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 21 12:39:46.986488 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985002    2556 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 21 12:39:46.986488 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985006    2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 21 12:39:46.986488 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985008    2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 21 12:39:46.986488 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985030    2556 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 21 12:39:46.986488 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985034    2556 feature_gate.go:328] unrecognized feature gate: Example
Jul 21 12:39:46.986488 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985037    2556 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 21 12:39:46.986960 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985040    2556 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 21 12:39:46.986960 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985042    2556 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 21 12:39:46.986960 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985047    2556 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 21 12:39:46.986960 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985051    2556 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 21 12:39:46.986960 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985055    2556 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 21 12:39:46.986960 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985057    2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 21 12:39:46.986960 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985060    2556 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 21 12:39:46.986960 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985064    2556 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 21 12:39:46.986960 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985066    2556 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 21 12:39:46.986960 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985068    2556 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 21 12:39:46.986960 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985071    2556 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 21 12:39:46.986960 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985074    2556 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 21 12:39:46.986960 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985076    2556 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 21 12:39:46.986960 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985079    2556 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 21 12:39:46.986960 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985081    2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 21 12:39:46.986960 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985083    2556 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 21 12:39:46.986960 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985086    2556 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 21 12:39:46.986960 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985088    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 21 12:39:46.986960 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985091    2556 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 21 12:39:46.987434 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985093    2556 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 21 12:39:46.987434 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985096    2556 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 21 12:39:46.987434 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985106    2556 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 21 12:39:46.987434 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985108    2556 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 21 12:39:46.987434 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985110    2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 21 12:39:46.987434 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985113    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 21 12:39:46.987434 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985115    2556 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 21 12:39:46.987434 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985118    2556 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 21 12:39:46.987434 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985120    2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 21 12:39:46.987434 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985123    2556 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 21 12:39:46.987434 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985125    2556 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 21 12:39:46.987434 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985127    2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 21 12:39:46.987434 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985130    2556 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 21 12:39:46.987434 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985133    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 21 12:39:46.987434 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985137    2556 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 21 12:39:46.987434 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985140    2556 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 21 12:39:46.987434 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985143    2556 feature_gate.go:328] unrecognized feature gate: Example2
Jul 21 12:39:46.987434 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985146    2556 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 21 12:39:46.987434 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985148    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 21 12:39:46.987911 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985151    2556 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 21 12:39:46.987911 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985154    2556 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 21 12:39:46.987911 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985157    2556 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 21 12:39:46.987911 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985159    2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 21 12:39:46.987911 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985162    2556 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 21 12:39:46.987911 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985165    2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 21 12:39:46.987911 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985167    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 21 12:39:46.987911 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985170    2556 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 21 12:39:46.987911 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985172    2556 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 21 12:39:46.987911 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985175    2556 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 21 12:39:46.987911 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985178    2556 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 21 12:39:46.987911 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985180    2556 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 21 12:39:46.987911 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985183    2556 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 21 12:39:46.987911 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985185    2556 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 21 12:39:46.987911 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985189    2556 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 21 12:39:46.987911 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985192    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 21 12:39:46.987911 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985200    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 21 12:39:46.987911 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985203    2556 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 21 12:39:46.987911 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985205    2556 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 21 12:39:46.987911 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985208    2556 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 21 12:39:46.988405 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985210    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 21 12:39:46.988405 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985213    2556 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 21 12:39:46.988405 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985215    2556 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 21 12:39:46.988405 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985218    2556 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 21 12:39:46.988405 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985220    2556 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 21 12:39:46.988405 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985223    2556 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 21 12:39:46.988405 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985226    2556 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 21 12:39:46.988405 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985228    2556 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 21 12:39:46.988405 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985231    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 21 12:39:46.988405 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985234    2556 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 21 12:39:46.988405 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985236    2556 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 21 12:39:46.988405 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985239    2556 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 21 12:39:46.988405 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985241    2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 21 12:39:46.988405 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985244    2556 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 21 12:39:46.988405 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985246    2556 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 21 12:39:46.988405 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985248    2556 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 21 12:39:46.988822 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.985253    2556 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 21 12:39:46.988822 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985393    2556 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 21 12:39:46.988822 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985399    2556 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 21 12:39:46.988822 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985402    2556 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 21 12:39:46.988822 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985405    2556 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 21 12:39:46.988822 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985407    2556 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 21 12:39:46.988822 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985410    2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 21 12:39:46.988822 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985413    2556 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 21 12:39:46.988822 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985416    2556 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 21 12:39:46.988822 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985418    2556 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 21 12:39:46.988822 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985421    2556 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 21 12:39:46.988822 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985424    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 21 12:39:46.988822 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985426    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 21 12:39:46.988822 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985444    2556 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 21 12:39:46.988822 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985447    2556 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 21 12:39:46.989203 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985450    2556 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 21 12:39:46.989203 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985452    2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 21 12:39:46.989203 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985455    2556 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 21 12:39:46.989203 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985458    2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 21 12:39:46.989203 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985460    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 21 12:39:46.989203 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985463    2556 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 21 12:39:46.989203 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985466    2556 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 21 12:39:46.989203 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985469    2556 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 21 12:39:46.989203 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985471    2556 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 21 12:39:46.989203 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985474    2556 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 21 12:39:46.989203 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985476    2556 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 21 12:39:46.989203 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985479    2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 21 12:39:46.989203 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985481    2556 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 21 12:39:46.989203 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985484    2556 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 21 12:39:46.989203 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985487    2556 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 21 12:39:46.989203 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985490    2556 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 21 12:39:46.989203 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985493    2556 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 21 12:39:46.989203 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985495    2556 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 21 12:39:46.989203 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985498    2556 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 21 12:39:46.989203 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985500    2556 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 21 12:39:46.989687 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985504    2556 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 21 12:39:46.989687 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985508    2556 feature_gate.go:328] unrecognized feature gate: Example
Jul 21 12:39:46.989687 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985511    2556 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 21 12:39:46.989687 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985513    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 21 12:39:46.989687 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985516    2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 21 12:39:46.989687 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985519    2556 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 21 12:39:46.989687 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985522    2556 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 21 12:39:46.989687 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985525    2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 21 12:39:46.989687 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985528    2556 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 21 12:39:46.989687 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985530    2556 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 21 12:39:46.989687 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985533    2556 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 21 12:39:46.989687 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985541    2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 21 12:39:46.989687 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985544    2556 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 21 12:39:46.989687 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985546    2556 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 21 12:39:46.989687 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985549    2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 21 12:39:46.989687 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985552    2556 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 21 12:39:46.989687 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985554    2556 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 21 12:39:46.989687 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985557    2556 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 21 12:39:46.989687 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985559    2556 feature_gate.go:328] unrecognized feature gate: Example2
Jul 21 12:39:46.989687 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985562    2556 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 21 12:39:46.990177 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985564    2556 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 21 12:39:46.990177 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985567    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 21 12:39:46.990177 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985569    2556 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 21 12:39:46.990177 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985571    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 21 12:39:46.990177 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985574    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 21 12:39:46.990177 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985576    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 21 12:39:46.990177 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985579    2556 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 21 12:39:46.990177 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985581    2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 21 12:39:46.990177 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985583    2556 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 21 12:39:46.990177 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985586    2556 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 21 12:39:46.990177 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985588    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 21 12:39:46.990177 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985591    2556 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 21 12:39:46.990177 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985593    2556 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 21 12:39:46.990177 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985596    2556 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 21 12:39:46.990177 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985598    2556 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 21 12:39:46.990177 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985600    2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 21 12:39:46.990177 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985603    2556 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 21 12:39:46.990177 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985605    2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 21 12:39:46.990177 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985608    2556 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 21 12:39:46.990631 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985611    2556 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 21 12:39:46.990631 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985615    2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 21 12:39:46.990631 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985617    2556 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 21 12:39:46.990631 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985619    2556 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 21 12:39:46.990631 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985622    2556 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 21 12:39:46.990631 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985631    2556 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 21 12:39:46.990631 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985633    2556 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 21 12:39:46.990631 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985636    2556 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 21 12:39:46.990631 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985639    2556 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 21 12:39:46.990631 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985643    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 21 12:39:46.990631 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985646    2556 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 21 12:39:46.990631 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985648    2556 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 21 12:39:46.990631 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985652    2556 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 21 12:39:46.990631 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985654    2556 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 21 12:39:46.990631 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985657    2556 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 21 12:39:46.990631 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985659    2556 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 21 12:39:46.990631 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985662    2556 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 21 12:39:46.990631 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985664    2556 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 21 12:39:46.990631 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985667    2556 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 21 12:39:46.991094 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985669    2556 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 21 12:39:46.991094 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985671    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 21 12:39:46.991094 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985673    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 21 12:39:46.991094 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985676    2556 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 21 12:39:46.991094 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985678    2556 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 21 12:39:46.991094 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985681    2556 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 21 12:39:46.991094 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:46.985683    2556 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 21 12:39:46.991094 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.985688    2556 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 21 12:39:46.991094 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.985827    2556 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 21 12:39:46.991094 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.989319    2556 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 21 12:39:46.991094 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.990693    2556 server.go:1019] "Starting client certificate rotation"
Jul 21 12:39:46.991094 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.990796    2556 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 21 12:39:46.991094 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.990853    2556 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 21 12:39:46.993457 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.993443    2556 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 21 12:39:46.995119 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:46.995103    2556 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 21 12:39:47.006387 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:47.006370    2556 log.go:25] "Validated CRI v1 runtime API"
Jul 21 12:39:47.009355 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:47.009339    2556 log.go:25] "Validated CRI v1 image API"
Jul 21 12:39:47.010153 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:47.010136    2556 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 21 12:39:47.012537 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:47.012518    2556 fs.go:135] Filesystem UUIDs: map[2b2fc9f3-7283-460e-aed2-c71c18c3c292:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 8a644e0e-f404-424b-8e79-145d7d93c5fb:/dev/nvme0n1p3]
Jul 21 12:39:47.012614 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:47.012535    2556 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 21 12:39:47.018302 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:47.018286    2556 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 21 12:39:47.018728 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:47.018611    2556 manager.go:216] Machine: {Timestamp:2026-07-21 12:39:47.017405156 +0000 UTC m=+0.321410262 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3103289 MemoryCapacity:32812175360 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2174afe9f38cadf4eb340b2951a4e0 SystemUUID:ec2174af-e9f3-8cad-f4eb-340b2951a4e0 BootID:1c03b5aa-1ee0-4827-b385-76d5edc90fcc Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:e5:9e:fd:59:7b Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:e5:9e:fd:59:7b Speed:0 Mtu:9001} {Name:ovs-system MacAddress:c2:8a:71:5a:99:fb Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812175360 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 21 12:39:47.018728 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:47.018726    2556 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 21 12:39:47.018830 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:47.018810    2556 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 21 12:39:47.022513 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:47.022312    2556 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 21 12:39:47.022653 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:47.022519    2556 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-130-2.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 21 12:39:47.023231 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:47.023220    2556 topology_manager.go:138] "Creating topology manager with none policy"
Jul 21 12:39:47.023277 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:47.023232    2556 container_manager_linux.go:309] "Creating device plugin manager"
Jul 21 12:39:47.023277 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:47.023240    2556 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 21 12:39:47.023913 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:47.023903    2556 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 21 12:39:47.025459 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:47.025448    2556 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 21 12:39:47.030104 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:47.030093    2556 state_mem.go:36] "Initialized new in-memory state store"
Jul 21 12:39:47.030219 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:47.030210    2556 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 21 12:39:47.030264 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:47.030238    2556 kubelet.go:486] "Attempting to sync node with API server"
Jul 21 12:39:47.030264 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:47.030248    2556 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 21 12:39:47.030264 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:47.030258    2556 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 21 12:39:47.030345 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:47.030267    2556 kubelet.go:398] "Adding apiserver pod source"
Jul 21 12:39:47.030345 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:47.030275    2556 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 21 12:39:47.030672 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:47.030657    2556 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 21 12:39:47.030749 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:47.030680    2556 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 21 12:39:47.032770 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:47.032755    2556 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 21 12:39:47.034974 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:47.034958    2556 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 21 12:39:47.035068 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:47.034996    2556 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 21 12:39:47.036264 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:47.036250    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 21 12:39:47.036358 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:47.036269    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 21 12:39:47.036358 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:47.036278    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 21 12:39:47.036358 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:47.036285    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 21 12:39:47.036358 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:47.036296    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 21 12:39:47.036358 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:47.036304    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 21 12:39:47.036358 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:47.036314    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 21 12:39:47.036358 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:47.036322    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 21 12:39:47.036358 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:47.036331    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 21 12:39:47.036358 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:47.036339    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 21 12:39:47.036358 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:47.036351    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 21 12:39:47.036358 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:47.036358    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 21 12:39:47.036818 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:47.036399    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 21 12:39:47.036818 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:47.036408    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 21 12:39:47.039628 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:39:47.039600    2556 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-130-2.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 21 12:39:47.039721 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:47.039699    2556 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-130-2.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 21 12:39:47.039780 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:39:47.039730    2556 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 21 12:39:47.039831 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:47.039802    2556 server.go:1268] "Started kubelet"
Jul 21 12:39:47.039917 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:47.039892    2556 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 21 12:39:47.039982 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:47.039933    2556 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 21 12:39:47.040043 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:47.040030    2556 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 21 12:39:47.040642 ip-10-0-130-2 systemd[1]: Started Kubernetes Kubelet.
Jul 21 12:39:47.041987 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:47.041968    2556 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 21 12:39:47.042482 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:47.042464    2556 server.go:310] "Adding debug handlers to kubelet server"
Jul 21 12:39:47.044896 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:39:47.044879    2556 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 21 12:39:47.045740 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:47.045723    2556 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-fcg98"
Jul 21 12:39:47.048630 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:39:47.047768    2556 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-2.ec2.internal.18c44e6e144f9f78  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-2.ec2.internal,UID:ip-10-0-130-2.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-130-2.ec2.internal,},FirstTimestamp:2026-07-21 12:39:47.039690616 +0000 UTC m=+0.343695701,LastTimestamp:2026-07-21 12:39:47.039690616 +0000 UTC m=+0.343695701,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-2.ec2.internal,}"
Jul 21 12:39:47.050862 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:47.050846    2556 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 21 12:39:47.050862 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:47.050859    2556 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 21 12:39:47.050987 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:39:47.050974    2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-2.ec2.internal\" not found"
Jul 21 12:39:47.051914 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:47.051526    2556 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 21 12:39:47.051914 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:47.051536    2556 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 21 12:39:47.051914 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:47.051550    2556 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 21 12:39:47.051914 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:47.051904    2556 reconstruct.go:97] "Volume reconstruction finished"
Jul 21 12:39:47.051914 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:47.051913    2556 reconciler.go:29] "Reconciler: start to sync state"
Jul 21 12:39:47.052531 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:47.052508    2556 factory.go:55] Registering systemd factory
Jul 21 12:39:47.052585 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:47.052545    2556 factory.go:223] Registration of the systemd container factory successfully
Jul 21 12:39:47.052750 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:47.052734    2556 factory.go:153] Registering CRI-O factory
Jul 21 12:39:47.052832 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:47.052752    2556 factory.go:223] Registration of the crio container factory successfully
Jul 21 12:39:47.052832 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:47.052821    2556 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 21 12:39:47.052928 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:47.052839    2556 factory.go:103] Registering Raw factory
Jul 21 12:39:47.052928 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:47.052855    2556 manager.go:1196] Started watching for new ooms in manager
Jul 21 12:39:47.053032 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:39:47.052946    2556 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 21 12:39:47.053032 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:39:47.052983    2556 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-130-2.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 21 12:39:47.053623 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:47.053607    2556 manager.go:349] Starting recovery of all containers
Jul 21 12:39:47.054905 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:47.054846    2556 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-fcg98"
Jul 21 12:39:47.066175 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:47.066157    2556 manager.go:354] Recovery completed
Jul 21 12:39:47.070024 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:47.069999    2556 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 21 12:39:47.071615 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:47.071595    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-2.ec2.internal" event="NodeHasSufficientMemory"
Jul 21 12:39:47.071673 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:47.071626    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-2.ec2.internal" event="NodeHasNoDiskPressure"
Jul 21 12:39:47.071673 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:47.071635    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-2.ec2.internal" event="NodeHasSufficientPID"
Jul 21 12:39:47.072176 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:47.072163    2556 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 21 12:39:47.072176 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:47.072173    2556 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 21 12:39:47.072321 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:47.072203    2556 state_mem.go:36] "Initialized new in-memory state store"
Jul 21 12:39:47.072321 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:47.072235    2556 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 21 12:39:47.072963 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:47.072941    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-2.ec2.internal" event="NodeHasSufficientMemory"
Jul 21 12:39:47.073061 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:47.072991    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-2.ec2.internal" event="NodeHasNoDiskPressure"
Jul 21 12:39:47.073061 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:47.073007    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-2.ec2.internal" event="NodeHasSufficientPID"
Jul 21 12:39:47.074298 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:47.074285    2556 policy_none.go:49] "None policy: Start"
Jul 21 12:39:47.074345 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:47.074302    2556 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 21 12:39:47.074345 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:47.074311    2556 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 21 12:39:47.076263 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:47.076250    2556 policy_none.go:47] "Start"
Jul 21 12:39:47.118637 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:47.118526    2556 manager.go:337] "Starting Device Plugin manager"
Jul 21 12:39:47.118716 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:39:47.118657    2556 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 21 12:39:47.118716 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:47.118669    2556 server.go:87] "Starting device plugin registration server"
Jul 21 12:39:47.118894 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:47.118882    2556 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 21 12:39:47.118937 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:47.118895    2556 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 21 12:39:47.119056 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:47.119035    2556 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 21 12:39:47.119160 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:47.119120    2556 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 21 12:39:47.119160 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:47.119130    2556 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 21 12:39:47.119802 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:39:47.119781    2556 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 21 12:39:47.119863 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:39:47.119836    2556 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-130-2.ec2.internal\" not found"
Jul 21 12:39:47.186278 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:47.186245    2556 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 21 12:39:47.188462 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:47.187526    2556 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 21 12:39:47.188462 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:47.187570    2556 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 21 12:39:47.188462 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:47.187596    2556 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 21 12:39:47.188462 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:39:47.187633    2556 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 21 12:39:47.189853 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:47.189837    2556 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 21 12:39:47.219655 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:47.219609    2556 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 21 12:39:47.220357 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:47.220342    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-2.ec2.internal" event="NodeHasSufficientMemory"
Jul 21 12:39:47.220427 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:47.220371    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-2.ec2.internal" event="NodeHasNoDiskPressure"
Jul 21 12:39:47.220427 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:47.220384    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-2.ec2.internal" event="NodeHasSufficientPID"
Jul 21 12:39:47.220427 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:47.220405    2556 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-130-2.ec2.internal"
Jul 21 12:39:47.229189 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:47.229169    2556 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-130-2.ec2.internal"
Jul 21 12:39:47.229236 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:39:47.229189    2556 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-130-2.ec2.internal\": node \"ip-10-0-130-2.ec2.internal\" not found"
Jul 21 12:39:47.253479 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:39:47.253461    2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-2.ec2.internal\" not found"
Jul 21 12:39:47.288596 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:47.288570    2556 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-2.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-130-2.ec2.internal"]
Jul 21 12:39:47.288652 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:47.288642    2556 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 21 12:39:47.290123 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:47.290109    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-2.ec2.internal" event="NodeHasSufficientMemory"
Jul 21 12:39:47.290188 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:47.290136    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-2.ec2.internal" event="NodeHasNoDiskPressure"
Jul 21 12:39:47.290188 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:47.290145    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-2.ec2.internal" event="NodeHasSufficientPID"
Jul 21 12:39:47.292445 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:47.292434    2556 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 21 12:39:47.292584 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:47.292570    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-2.ec2.internal"
Jul 21 12:39:47.292626 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:47.292601    2556 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 21 12:39:47.293146 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:47.293131    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-2.ec2.internal" event="NodeHasSufficientMemory"
Jul 21 12:39:47.293146 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:47.293142    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-2.ec2.internal" event="NodeHasSufficientMemory"
Jul 21 12:39:47.293255 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:47.293157    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-2.ec2.internal" event="NodeHasNoDiskPressure"
Jul 21 12:39:47.293255 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:47.293165    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-2.ec2.internal" event="NodeHasNoDiskPressure"
Jul 21 12:39:47.293255 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:47.293168    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-2.ec2.internal" event="NodeHasSufficientPID"
Jul 21 12:39:47.293255 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:47.293184    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-2.ec2.internal" event="NodeHasSufficientPID"
Jul 21 12:39:47.295521 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:47.295505    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-2.ec2.internal"
Jul 21 12:39:47.295585 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:47.295544    2556 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 21 12:39:47.296643 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:47.296624    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-2.ec2.internal" event="NodeHasSufficientMemory"
Jul 21 12:39:47.296737 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:47.296657    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-2.ec2.internal" event="NodeHasNoDiskPressure"
Jul 21 12:39:47.296737 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:47.296670    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-2.ec2.internal" event="NodeHasSufficientPID"
Jul 21 12:39:47.311522 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:39:47.311500    2556 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-2.ec2.internal\" not found" node="ip-10-0-130-2.ec2.internal"
Jul 21 12:39:47.315778 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:39:47.315759    2556 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-2.ec2.internal\" not found" node="ip-10-0-130-2.ec2.internal"
Jul 21 12:39:47.352723 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:47.352701    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/1564417db20202d90def33f3db6f1c9c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-2.ec2.internal\" (UID: \"1564417db20202d90def33f3db6f1c9c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-2.ec2.internal"
Jul 21 12:39:47.352788 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:47.352737    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1564417db20202d90def33f3db6f1c9c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-2.ec2.internal\" (UID: \"1564417db20202d90def33f3db6f1c9c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-2.ec2.internal"
Jul 21 12:39:47.352788 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:47.352772    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/88a4b370840ce6eb63d3d3c786c424a4-config\") pod \"kube-apiserver-proxy-ip-10-0-130-2.ec2.internal\" (UID: \"88a4b370840ce6eb63d3d3c786c424a4\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-2.ec2.internal"
Jul 21 12:39:47.353594 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:39:47.353579    2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-2.ec2.internal\" not found"
Jul 21 12:39:47.453159 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:47.453139    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1564417db20202d90def33f3db6f1c9c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-2.ec2.internal\" (UID: \"1564417db20202d90def33f3db6f1c9c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-2.ec2.internal"
Jul 21 12:39:47.453216 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:47.453162    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/88a4b370840ce6eb63d3d3c786c424a4-config\") pod \"kube-apiserver-proxy-ip-10-0-130-2.ec2.internal\" (UID: \"88a4b370840ce6eb63d3d3c786c424a4\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-2.ec2.internal"
Jul 21 12:39:47.453216 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:47.453191    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/1564417db20202d90def33f3db6f1c9c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-2.ec2.internal\" (UID: \"1564417db20202d90def33f3db6f1c9c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-2.ec2.internal"
Jul 21 12:39:47.453289 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:47.453220    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/1564417db20202d90def33f3db6f1c9c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-2.ec2.internal\" (UID: \"1564417db20202d90def33f3db6f1c9c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-2.ec2.internal"
Jul 21 12:39:47.453289 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:47.453229    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1564417db20202d90def33f3db6f1c9c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-2.ec2.internal\" (UID: \"1564417db20202d90def33f3db6f1c9c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-2.ec2.internal"
Jul 21 12:39:47.453289 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:47.453242    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/88a4b370840ce6eb63d3d3c786c424a4-config\") pod \"kube-apiserver-proxy-ip-10-0-130-2.ec2.internal\" (UID: \"88a4b370840ce6eb63d3d3c786c424a4\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-2.ec2.internal"
Jul 21 12:39:47.454207 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:39:47.454191    2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-2.ec2.internal\" not found"
Jul 21 12:39:47.554515 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:39:47.554493    2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-2.ec2.internal\" not found"
Jul 21 12:39:47.615703 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:47.615666    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-2.ec2.internal"
Jul 21 12:39:47.619598 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:47.619582    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-2.ec2.internal"
Jul 21 12:39:47.654792 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:39:47.654765    2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-2.ec2.internal\" not found"
Jul 21 12:39:47.755263 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:39:47.755235    2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-2.ec2.internal\" not found"
Jul 21 12:39:47.855767 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:39:47.855711    2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-2.ec2.internal\" not found"
Jul 21 12:39:47.956146 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:39:47.956126    2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-2.ec2.internal\" not found"
Jul 21 12:39:47.990322 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:47.990291    2556 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 21 12:39:47.990707 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:47.990429    2556 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 21 12:39:48.051618 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:48.051581    2556 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 21 12:39:48.056542 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:39:48.056521    2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-2.ec2.internal\" not found"
Jul 21 12:39:48.060558 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:48.060531    2556 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-20 12:34:47 +0000 UTC" deadline="2028-04-22 15:45:19.8721615 +0000 UTC"
Jul 21 12:39:48.060644 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:48.060558    2556 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15387h5m31.811607754s"
Jul 21 12:39:48.156308 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:48.156288    2556 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 21 12:39:48.157477 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:39:48.157461    2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-2.ec2.internal\" not found"
Jul 21 12:39:48.258135 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:39:48.258110    2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-2.ec2.internal\" not found"
Jul 21 12:39:48.278793 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:48.278764    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod88a4b370840ce6eb63d3d3c786c424a4.slice/crio-9fac6f29392e9ee6e8615e01c66cc871f485ff221a00f39022fc083f171efd16 WatchSource:0}: Error finding container 9fac6f29392e9ee6e8615e01c66cc871f485ff221a00f39022fc083f171efd16: Status 404 returned error can't find the container with id 9fac6f29392e9ee6e8615e01c66cc871f485ff221a00f39022fc083f171efd16
Jul 21 12:39:48.279207 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:48.279182    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1564417db20202d90def33f3db6f1c9c.slice/crio-d8125c07d247c88e70c89709995be350b5debdec6c402a9735fbe1230ee0a28c WatchSource:0}: Error finding container d8125c07d247c88e70c89709995be350b5debdec6c402a9735fbe1230ee0a28c: Status 404 returned error can't find the container with id d8125c07d247c88e70c89709995be350b5debdec6c402a9735fbe1230ee0a28c
Jul 21 12:39:48.282851 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:48.282836    2556 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 21 12:39:48.358712 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:39:48.358689    2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-2.ec2.internal\" not found"
Jul 21 12:39:48.379177 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:48.379156    2556 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 21 12:39:48.452925 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:48.452873    2556 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-2.ec2.internal"
Jul 21 12:39:48.466053 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:48.466027    2556 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 21 12:39:48.467122 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:48.467109    2556 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-2.ec2.internal"
Jul 21 12:39:48.477308 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:48.477289    2556 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 21 12:39:49.031270 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.030960    2556 apiserver.go:52] "Watching apiserver"
Jul 21 12:39:49.039213 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.038801    2556 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 21 12:39:49.040636 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.040233    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/konnectivity-agent-tpxr6","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xq6pc","openshift-cluster-node-tuning-operator/tuned-4cq9n","openshift-dns/node-resolver-z24v8","openshift-multus/multus-additional-cni-plugins-vbvlg","openshift-multus/network-metrics-daemon-jfqn8","openshift-network-diagnostics/network-check-target-clxmp","openshift-ovn-kubernetes/ovnkube-node-488zb","kube-system/global-pull-secret-syncer-mchwp","kube-system/kube-apiserver-proxy-ip-10-0-130-2.ec2.internal","openshift-image-registry/node-ca-76wr9","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-2.ec2.internal","openshift-multus/multus-nwthp","openshift-network-operator/iptables-alerter-h6bkx"]
Jul 21 12:39:49.043399 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.042940    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-clxmp"
Jul 21 12:39:49.043399 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:39:49.043020    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-clxmp" podUID="37f02e4b-2a9f-4e44-8c74-d2428bd97ee5"
Jul 21 12:39:49.046313 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.046287    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xq6pc"
Jul 21 12:39:49.049908 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.049884    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-tvsqj\""
Jul 21 12:39:49.052204 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.052186    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 21 12:39:49.052486 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.052460    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 21 12:39:49.052486 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.052483    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 21 12:39:49.052685 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.052667    2556 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 21 12:39:49.055132 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.055109    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-4cq9n"
Jul 21 12:39:49.057542 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.057475    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-tpxr6"
Jul 21 12:39:49.057637 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.057571    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-wx662\""
Jul 21 12:39:49.057779 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.057757    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 21 12:39:49.057993 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.057949    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 21 12:39:49.059799 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.059779    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-vbvlg"
Jul 21 12:39:49.059994 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.059954    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 21 12:39:49.060337 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.060311    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-xd589\""
Jul 21 12:39:49.060487 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.060457    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 21 12:39:49.062596 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.061652    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/38932224-eb5c-47f9-8515-8d4ba212808d-etc-selinux\") pod \"aws-ebs-csi-driver-node-xq6pc\" (UID: \"38932224-eb5c-47f9-8515-8d4ba212808d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xq6pc"
Jul 21 12:39:49.062596 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.061691    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b58bf088-1be7-436e-af10-a747c9f968fb-etc-modprobe-d\") pod \"tuned-4cq9n\" (UID: \"b58bf088-1be7-436e-af10-a747c9f968fb\") " pod="openshift-cluster-node-tuning-operator/tuned-4cq9n"
Jul 21 12:39:49.062596 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.061720    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mbfzx\" (UniqueName: \"kubernetes.io/projected/37f02e4b-2a9f-4e44-8c74-d2428bd97ee5-kube-api-access-mbfzx\") pod \"network-check-target-clxmp\" (UID: \"37f02e4b-2a9f-4e44-8c74-d2428bd97ee5\") " pod="openshift-network-diagnostics/network-check-target-clxmp"
Jul 21 12:39:49.062596 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.061749    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/38932224-eb5c-47f9-8515-8d4ba212808d-socket-dir\") pod \"aws-ebs-csi-driver-node-xq6pc\" (UID: \"38932224-eb5c-47f9-8515-8d4ba212808d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xq6pc"
Jul 21 12:39:49.062596 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.061773    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/38932224-eb5c-47f9-8515-8d4ba212808d-registration-dir\") pod \"aws-ebs-csi-driver-node-xq6pc\" (UID: \"38932224-eb5c-47f9-8515-8d4ba212808d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xq6pc"
Jul 21 12:39:49.062596 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.061801    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g5bnc\" (UniqueName: \"kubernetes.io/projected/38932224-eb5c-47f9-8515-8d4ba212808d-kube-api-access-g5bnc\") pod \"aws-ebs-csi-driver-node-xq6pc\" (UID: \"38932224-eb5c-47f9-8515-8d4ba212808d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xq6pc"
Jul 21 12:39:49.062596 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.061826    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b58bf088-1be7-436e-af10-a747c9f968fb-lib-modules\") pod \"tuned-4cq9n\" (UID: \"b58bf088-1be7-436e-af10-a747c9f968fb\") " pod="openshift-cluster-node-tuning-operator/tuned-4cq9n"
Jul 21 12:39:49.062596 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.061850    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b58bf088-1be7-436e-af10-a747c9f968fb-var-lib-kubelet\") pod \"tuned-4cq9n\" (UID: \"b58bf088-1be7-436e-af10-a747c9f968fb\") " pod="openshift-cluster-node-tuning-operator/tuned-4cq9n"
Jul 21 12:39:49.062596 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.061873    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b58bf088-1be7-436e-af10-a747c9f968fb-host\") pod \"tuned-4cq9n\" (UID: \"b58bf088-1be7-436e-af10-a747c9f968fb\") " pod="openshift-cluster-node-tuning-operator/tuned-4cq9n"
Jul 21 12:39:49.062596 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.061898    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jxlpq\" (UniqueName: \"kubernetes.io/projected/b58bf088-1be7-436e-af10-a747c9f968fb-kube-api-access-jxlpq\") pod \"tuned-4cq9n\" (UID: \"b58bf088-1be7-436e-af10-a747c9f968fb\") " pod="openshift-cluster-node-tuning-operator/tuned-4cq9n"
Jul 21 12:39:49.062596 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.061922    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/38932224-eb5c-47f9-8515-8d4ba212808d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-xq6pc\" (UID: \"38932224-eb5c-47f9-8515-8d4ba212808d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xq6pc"
Jul 21 12:39:49.062596 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.061947    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b58bf088-1be7-436e-af10-a747c9f968fb-etc-sysconfig\") pod \"tuned-4cq9n\" (UID: \"b58bf088-1be7-436e-af10-a747c9f968fb\") " pod="openshift-cluster-node-tuning-operator/tuned-4cq9n"
Jul 21 12:39:49.062596 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.061970    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b58bf088-1be7-436e-af10-a747c9f968fb-etc-sysctl-d\") pod \"tuned-4cq9n\" (UID: \"b58bf088-1be7-436e-af10-a747c9f968fb\") " pod="openshift-cluster-node-tuning-operator/tuned-4cq9n"
Jul 21 12:39:49.062596 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.061993    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b58bf088-1be7-436e-af10-a747c9f968fb-etc-sysctl-conf\") pod \"tuned-4cq9n\" (UID: \"b58bf088-1be7-436e-af10-a747c9f968fb\") " pod="openshift-cluster-node-tuning-operator/tuned-4cq9n"
Jul 21 12:39:49.062596 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.062027    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b58bf088-1be7-436e-af10-a747c9f968fb-etc-systemd\") pod \"tuned-4cq9n\" (UID: \"b58bf088-1be7-436e-af10-a747c9f968fb\") " pod="openshift-cluster-node-tuning-operator/tuned-4cq9n"
Jul 21 12:39:49.062596 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.062069    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b58bf088-1be7-436e-af10-a747c9f968fb-run\") pod \"tuned-4cq9n\" (UID: \"b58bf088-1be7-436e-af10-a747c9f968fb\") " pod="openshift-cluster-node-tuning-operator/tuned-4cq9n"
Jul 21 12:39:49.063581 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.062092    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b58bf088-1be7-436e-af10-a747c9f968fb-sys\") pod \"tuned-4cq9n\" (UID: \"b58bf088-1be7-436e-af10-a747c9f968fb\") " pod="openshift-cluster-node-tuning-operator/tuned-4cq9n"
Jul 21 12:39:49.063581 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.062116    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/38932224-eb5c-47f9-8515-8d4ba212808d-device-dir\") pod \"aws-ebs-csi-driver-node-xq6pc\" (UID: \"38932224-eb5c-47f9-8515-8d4ba212808d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xq6pc"
Jul 21 12:39:49.063581 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.062142    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/38932224-eb5c-47f9-8515-8d4ba212808d-sys-fs\") pod \"aws-ebs-csi-driver-node-xq6pc\" (UID: \"38932224-eb5c-47f9-8515-8d4ba212808d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xq6pc"
Jul 21 12:39:49.063581 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.062177    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b58bf088-1be7-436e-af10-a747c9f968fb-etc-kubernetes\") pod \"tuned-4cq9n\" (UID: \"b58bf088-1be7-436e-af10-a747c9f968fb\") " pod="openshift-cluster-node-tuning-operator/tuned-4cq9n"
Jul 21 12:39:49.063581 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.062203    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b58bf088-1be7-436e-af10-a747c9f968fb-etc-tuned\") pod \"tuned-4cq9n\" (UID: \"b58bf088-1be7-436e-af10-a747c9f968fb\") " pod="openshift-cluster-node-tuning-operator/tuned-4cq9n"
Jul 21 12:39:49.063581 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.062242    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b58bf088-1be7-436e-af10-a747c9f968fb-tmp\") pod \"tuned-4cq9n\" (UID: \"b58bf088-1be7-436e-af10-a747c9f968fb\") " pod="openshift-cluster-node-tuning-operator/tuned-4cq9n"
Jul 21 12:39:49.063581 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.062345    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 21 12:39:49.063581 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.062749    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-lw4cj\""
Jul 21 12:39:49.063581 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.063056    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-488zb"
Jul 21 12:39:49.063581 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.063147    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 21 12:39:49.063581 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.063349    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 21 12:39:49.063581 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.063541    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 21 12:39:49.070179 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.065435    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 21 12:39:49.070179 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.068197    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 21 12:39:49.070179 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.068621    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-jh5w6\""
Jul 21 12:39:49.070179 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.069221    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-z24v8"
Jul 21 12:39:49.070680 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.070656    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jfqn8"
Jul 21 12:39:49.070839 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:39:49.070810    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jfqn8" podUID="543737a1-6fd3-489b-a2b2-9822825c0007"
Jul 21 12:39:49.071017 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.070995    2556 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 21 12:39:49.073110 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.072342    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-vm5ss\""
Jul 21 12:39:49.073110 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.072617    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 21 12:39:49.073110 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.072971    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 21 12:39:49.073897 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.073558    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mchwp"
Jul 21 12:39:49.073897 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:39:49.073631    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mchwp" podUID="d71014d4-ae0b-437e-adca-886a8ff5ac7f"
Jul 21 12:39:49.073897 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.073751    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 21 12:39:49.074129 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.074116    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 21 12:39:49.074408 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.074298    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 21 12:39:49.076609 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.076589    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 21 12:39:49.077960 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.076834    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 21 12:39:49.077960 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.077632    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-76wr9"
Jul 21 12:39:49.080872 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.080479    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 21 12:39:49.080872 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.080555    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 21 12:39:49.080872 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.080713    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 21 12:39:49.081282 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.081266    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-cfl8l\""
Jul 21 12:39:49.081842 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.081562    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-nwthp"
Jul 21 12:39:49.083734 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.083714    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 21 12:39:49.084395 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.084380    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-h6bkx"
Jul 21 12:39:49.085149 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.085125    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-stjnz\""
Jul 21 12:39:49.086263 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.086198    2556 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-272fl"
Jul 21 12:39:49.087058 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.087039    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 21 12:39:49.087297 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.087281    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 21 12:39:49.087519 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.087505    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 21 12:39:49.087987 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.087968    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-2jhhq\""
Jul 21 12:39:49.101232 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.100851    2556 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-272fl"
Jul 21 12:39:49.153273 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.153254    2556 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 21 12:39:49.162630 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.162606    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/2c5a6a09-909c-4e4c-a05c-7d8f962bb20e-run-ovn\") pod \"ovnkube-node-488zb\" (UID: \"2c5a6a09-909c-4e4c-a05c-7d8f962bb20e\") " pod="openshift-ovn-kubernetes/ovnkube-node-488zb"
Jul 21 12:39:49.162747 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.162649    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/38932224-eb5c-47f9-8515-8d4ba212808d-socket-dir\") pod \"aws-ebs-csi-driver-node-xq6pc\" (UID: \"38932224-eb5c-47f9-8515-8d4ba212808d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xq6pc"
Jul 21 12:39:49.162747 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.162681    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g5bnc\" (UniqueName: \"kubernetes.io/projected/38932224-eb5c-47f9-8515-8d4ba212808d-kube-api-access-g5bnc\") pod \"aws-ebs-csi-driver-node-xq6pc\" (UID: \"38932224-eb5c-47f9-8515-8d4ba212808d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xq6pc"
Jul 21 12:39:49.162747 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.162706    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/2c5a6a09-909c-4e4c-a05c-7d8f962bb20e-systemd-units\") pod \"ovnkube-node-488zb\" (UID: \"2c5a6a09-909c-4e4c-a05c-7d8f962bb20e\") " pod="openshift-ovn-kubernetes/ovnkube-node-488zb"
Jul 21 12:39:49.162747 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.162734    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2c5a6a09-909c-4e4c-a05c-7d8f962bb20e-host-slash\") pod \"ovnkube-node-488zb\" (UID: \"2c5a6a09-909c-4e4c-a05c-7d8f962bb20e\") " pod="openshift-ovn-kubernetes/ovnkube-node-488zb"
Jul 21 12:39:49.162950 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.162868    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/38932224-eb5c-47f9-8515-8d4ba212808d-socket-dir\") pod \"aws-ebs-csi-driver-node-xq6pc\" (UID: \"38932224-eb5c-47f9-8515-8d4ba212808d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xq6pc"
Jul 21 12:39:49.162950 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.162915    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6glhm\" (UniqueName: \"kubernetes.io/projected/543737a1-6fd3-489b-a2b2-9822825c0007-kube-api-access-6glhm\") pod \"network-metrics-daemon-jfqn8\" (UID: \"543737a1-6fd3-489b-a2b2-9822825c0007\") " pod="openshift-multus/network-metrics-daemon-jfqn8"
Jul 21 12:39:49.163083 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.162952    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/0f25175e-7fc8-497f-88f1-42b3d583bce2-konnectivity-ca\") pod \"konnectivity-agent-tpxr6\" (UID: \"0f25175e-7fc8-497f-88f1-42b3d583bce2\") " pod="kube-system/konnectivity-agent-tpxr6"
Jul 21 12:39:49.163083 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.162979    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e5faf5b4-5f7e-44dd-9503-3cc15f07533e-tmp-dir\") pod \"node-resolver-z24v8\" (UID: \"e5faf5b4-5f7e-44dd-9503-3cc15f07533e\") " pod="openshift-dns/node-resolver-z24v8"
Jul 21 12:39:49.163083 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.163014    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jxlpq\" (UniqueName: \"kubernetes.io/projected/b58bf088-1be7-436e-af10-a747c9f968fb-kube-api-access-jxlpq\") pod \"tuned-4cq9n\" (UID: \"b58bf088-1be7-436e-af10-a747c9f968fb\") " pod="openshift-cluster-node-tuning-operator/tuned-4cq9n"
Jul 21 12:39:49.163083 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.163043    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/38932224-eb5c-47f9-8515-8d4ba212808d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-xq6pc\" (UID: \"38932224-eb5c-47f9-8515-8d4ba212808d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xq6pc"
Jul 21 12:39:49.163083 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.163065    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c2aa47a3-b6ce-4807-8e97-aaf79dc1d941-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-vbvlg\" (UID: \"c2aa47a3-b6ce-4807-8e97-aaf79dc1d941\") " pod="openshift-multus/multus-additional-cni-plugins-vbvlg"
Jul 21 12:39:49.163318 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.163106    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2c5a6a09-909c-4e4c-a05c-7d8f962bb20e-host-run-netns\") pod \"ovnkube-node-488zb\" (UID: \"2c5a6a09-909c-4e4c-a05c-7d8f962bb20e\") " pod="openshift-ovn-kubernetes/ovnkube-node-488zb"
Jul 21 12:39:49.163318 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.163165    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/38932224-eb5c-47f9-8515-8d4ba212808d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-xq6pc\" (UID: \"38932224-eb5c-47f9-8515-8d4ba212808d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xq6pc"
Jul 21 12:39:49.163318 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.163213    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d71014d4-ae0b-437e-adca-886a8ff5ac7f-kubelet-config\") pod \"global-pull-secret-syncer-mchwp\" (UID: \"d71014d4-ae0b-437e-adca-886a8ff5ac7f\") " pod="kube-system/global-pull-secret-syncer-mchwp"
Jul 21 12:39:49.163318 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.163257    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xt6df\" (UniqueName: \"kubernetes.io/projected/d108f652-91d5-4bd3-9a81-a24fccd7c8a3-kube-api-access-xt6df\") pod \"multus-nwthp\" (UID: \"d108f652-91d5-4bd3-9a81-a24fccd7c8a3\") " pod="openshift-multus/multus-nwthp"
Jul 21 12:39:49.163554 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.163313    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b58bf088-1be7-436e-af10-a747c9f968fb-etc-sysctl-d\") pod \"tuned-4cq9n\" (UID: \"b58bf088-1be7-436e-af10-a747c9f968fb\") " pod="openshift-cluster-node-tuning-operator/tuned-4cq9n"
Jul 21 12:39:49.163554 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.163347    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b58bf088-1be7-436e-af10-a747c9f968fb-etc-systemd\") pod \"tuned-4cq9n\" (UID: \"b58bf088-1be7-436e-af10-a747c9f968fb\") " pod="openshift-cluster-node-tuning-operator/tuned-4cq9n"
Jul 21 12:39:49.163554 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.163379    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-82v6h\" (UniqueName: \"kubernetes.io/projected/2c5a6a09-909c-4e4c-a05c-7d8f962bb20e-kube-api-access-82v6h\") pod \"ovnkube-node-488zb\" (UID: \"2c5a6a09-909c-4e4c-a05c-7d8f962bb20e\") " pod="openshift-ovn-kubernetes/ovnkube-node-488zb"
Jul 21 12:39:49.163554 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.163409    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d108f652-91d5-4bd3-9a81-a24fccd7c8a3-host-run-netns\") pod \"multus-nwthp\" (UID: \"d108f652-91d5-4bd3-9a81-a24fccd7c8a3\") " pod="openshift-multus/multus-nwthp"
Jul 21 12:39:49.163554 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.163429    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b58bf088-1be7-436e-af10-a747c9f968fb-etc-systemd\") pod \"tuned-4cq9n\" (UID: \"b58bf088-1be7-436e-af10-a747c9f968fb\") " pod="openshift-cluster-node-tuning-operator/tuned-4cq9n"
Jul 21 12:39:49.163554 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.163435    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c2aa47a3-b6ce-4807-8e97-aaf79dc1d941-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-vbvlg\" (UID: \"c2aa47a3-b6ce-4807-8e97-aaf79dc1d941\") " pod="openshift-multus/multus-additional-cni-plugins-vbvlg"
Jul 21 12:39:49.163554 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.163444    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b58bf088-1be7-436e-af10-a747c9f968fb-etc-sysctl-d\") pod \"tuned-4cq9n\" (UID: \"b58bf088-1be7-436e-af10-a747c9f968fb\") " pod="openshift-cluster-node-tuning-operator/tuned-4cq9n"
Jul 21 12:39:49.163554 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.163461    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/2c5a6a09-909c-4e4c-a05c-7d8f962bb20e-host-kubelet\") pod \"ovnkube-node-488zb\" (UID: \"2c5a6a09-909c-4e4c-a05c-7d8f962bb20e\") " pod="openshift-ovn-kubernetes/ovnkube-node-488zb"
Jul 21 12:39:49.163554 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.163489    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2c5a6a09-909c-4e4c-a05c-7d8f962bb20e-run-openvswitch\") pod \"ovnkube-node-488zb\" (UID: \"2c5a6a09-909c-4e4c-a05c-7d8f962bb20e\") " pod="openshift-ovn-kubernetes/ovnkube-node-488zb"
Jul 21 12:39:49.163554 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.163513    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/2c5a6a09-909c-4e4c-a05c-7d8f962bb20e-node-log\") pod \"ovnkube-node-488zb\" (UID: \"2c5a6a09-909c-4e4c-a05c-7d8f962bb20e\") " pod="openshift-ovn-kubernetes/ovnkube-node-488zb"
Jul 21 12:39:49.163554 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.163537    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5adf7abd-fea9-4a2a-8f96-a73bd0a3404a-serviceca\") pod \"node-ca-76wr9\" (UID: \"5adf7abd-fea9-4a2a-8f96-a73bd0a3404a\") " pod="openshift-image-registry/node-ca-76wr9"
Jul 21 12:39:49.164039 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.163563    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d108f652-91d5-4bd3-9a81-a24fccd7c8a3-multus-conf-dir\") pod \"multus-nwthp\" (UID: \"d108f652-91d5-4bd3-9a81-a24fccd7c8a3\") " pod="openshift-multus/multus-nwthp"
Jul 21 12:39:49.164039 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.163609    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hkq64\" (UniqueName: \"kubernetes.io/projected/2b732ef7-474f-4e5e-92b8-03ad2677f408-kube-api-access-hkq64\") pod \"iptables-alerter-h6bkx\" (UID: \"2b732ef7-474f-4e5e-92b8-03ad2677f408\") " pod="openshift-network-operator/iptables-alerter-h6bkx"
Jul 21 12:39:49.164039 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.163653    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/38932224-eb5c-47f9-8515-8d4ba212808d-registration-dir\") pod \"aws-ebs-csi-driver-node-xq6pc\" (UID: \"38932224-eb5c-47f9-8515-8d4ba212808d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xq6pc"
Jul 21 12:39:49.164039 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.163712    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/38932224-eb5c-47f9-8515-8d4ba212808d-registration-dir\") pod \"aws-ebs-csi-driver-node-xq6pc\" (UID: \"38932224-eb5c-47f9-8515-8d4ba212808d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xq6pc"
Jul 21 12:39:49.164039 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.163747    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c2aa47a3-b6ce-4807-8e97-aaf79dc1d941-os-release\") pod \"multus-additional-cni-plugins-vbvlg\" (UID: \"c2aa47a3-b6ce-4807-8e97-aaf79dc1d941\") " pod="openshift-multus/multus-additional-cni-plugins-vbvlg"
Jul 21 12:39:49.164039 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.163771    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/2c5a6a09-909c-4e4c-a05c-7d8f962bb20e-log-socket\") pod \"ovnkube-node-488zb\" (UID: \"2c5a6a09-909c-4e4c-a05c-7d8f962bb20e\") " pod="openshift-ovn-kubernetes/ovnkube-node-488zb"
Jul 21 12:39:49.164039 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.163794    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/2c5a6a09-909c-4e4c-a05c-7d8f962bb20e-host-cni-netd\") pod \"ovnkube-node-488zb\" (UID: \"2c5a6a09-909c-4e4c-a05c-7d8f962bb20e\") " pod="openshift-ovn-kubernetes/ovnkube-node-488zb"
Jul 21 12:39:49.164039 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.163820    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/2c5a6a09-909c-4e4c-a05c-7d8f962bb20e-env-overrides\") pod \"ovnkube-node-488zb\" (UID: \"2c5a6a09-909c-4e4c-a05c-7d8f962bb20e\") " pod="openshift-ovn-kubernetes/ovnkube-node-488zb"
Jul 21 12:39:49.164039 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.163850    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5adf7abd-fea9-4a2a-8f96-a73bd0a3404a-host\") pod \"node-ca-76wr9\" (UID: \"5adf7abd-fea9-4a2a-8f96-a73bd0a3404a\") " pod="openshift-image-registry/node-ca-76wr9"
Jul 21 12:39:49.164039 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.163874    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/d108f652-91d5-4bd3-9a81-a24fccd7c8a3-hostroot\") pod \"multus-nwthp\" (UID: \"d108f652-91d5-4bd3-9a81-a24fccd7c8a3\") " pod="openshift-multus/multus-nwthp"
Jul 21 12:39:49.164039 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.163898    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/d108f652-91d5-4bd3-9a81-a24fccd7c8a3-host-run-multus-certs\") pod \"multus-nwthp\" (UID: \"d108f652-91d5-4bd3-9a81-a24fccd7c8a3\") " pod="openshift-multus/multus-nwthp"
Jul 21 12:39:49.164039 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.163924    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b58bf088-1be7-436e-af10-a747c9f968fb-var-lib-kubelet\") pod \"tuned-4cq9n\" (UID: \"b58bf088-1be7-436e-af10-a747c9f968fb\") " pod="openshift-cluster-node-tuning-operator/tuned-4cq9n"
Jul 21 12:39:49.164039 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.164028    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b58bf088-1be7-436e-af10-a747c9f968fb-host\") pod \"tuned-4cq9n\" (UID: \"b58bf088-1be7-436e-af10-a747c9f968fb\") " pod="openshift-cluster-node-tuning-operator/tuned-4cq9n"
Jul 21 12:39:49.164683 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.164083    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2c5a6a09-909c-4e4c-a05c-7d8f962bb20e-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-488zb\" (UID: \"2c5a6a09-909c-4e4c-a05c-7d8f962bb20e\") " pod="openshift-ovn-kubernetes/ovnkube-node-488zb"
Jul 21 12:39:49.164683 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.164085    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b58bf088-1be7-436e-af10-a747c9f968fb-var-lib-kubelet\") pod \"tuned-4cq9n\" (UID: \"b58bf088-1be7-436e-af10-a747c9f968fb\") " pod="openshift-cluster-node-tuning-operator/tuned-4cq9n"
Jul 21 12:39:49.164683 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.164141    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d108f652-91d5-4bd3-9a81-a24fccd7c8a3-system-cni-dir\") pod \"multus-nwthp\" (UID: \"d108f652-91d5-4bd3-9a81-a24fccd7c8a3\") " pod="openshift-multus/multus-nwthp"
Jul 21 12:39:49.164683 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.164156    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b58bf088-1be7-436e-af10-a747c9f968fb-host\") pod \"tuned-4cq9n\" (UID: \"b58bf088-1be7-436e-af10-a747c9f968fb\") " pod="openshift-cluster-node-tuning-operator/tuned-4cq9n"
Jul 21 12:39:49.164683 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.164170    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d108f652-91d5-4bd3-9a81-a24fccd7c8a3-os-release\") pod \"multus-nwthp\" (UID: \"d108f652-91d5-4bd3-9a81-a24fccd7c8a3\") " pod="openshift-multus/multus-nwthp"
Jul 21 12:39:49.164683 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.164196    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d108f652-91d5-4bd3-9a81-a24fccd7c8a3-cnibin\") pod \"multus-nwthp\" (UID: \"d108f652-91d5-4bd3-9a81-a24fccd7c8a3\") " pod="openshift-multus/multus-nwthp"
Jul 21 12:39:49.164683 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.164229    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d108f652-91d5-4bd3-9a81-a24fccd7c8a3-cni-binary-copy\") pod \"multus-nwthp\" (UID: \"d108f652-91d5-4bd3-9a81-a24fccd7c8a3\") " pod="openshift-multus/multus-nwthp"
Jul 21 12:39:49.164683 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.164263    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d108f652-91d5-4bd3-9a81-a24fccd7c8a3-host-var-lib-kubelet\") pod \"multus-nwthp\" (UID: \"d108f652-91d5-4bd3-9a81-a24fccd7c8a3\") " pod="openshift-multus/multus-nwthp"
Jul 21 12:39:49.164683 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.164293    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b58bf088-1be7-436e-af10-a747c9f968fb-etc-kubernetes\") pod \"tuned-4cq9n\" (UID: \"b58bf088-1be7-436e-af10-a747c9f968fb\") " pod="openshift-cluster-node-tuning-operator/tuned-4cq9n"
Jul 21 12:39:49.164683 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.164319    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/0f25175e-7fc8-497f-88f1-42b3d583bce2-agent-certs\") pod \"konnectivity-agent-tpxr6\" (UID: \"0f25175e-7fc8-497f-88f1-42b3d583bce2\") " pod="kube-system/konnectivity-agent-tpxr6"
Jul 21 12:39:49.164683 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.164344    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d108f652-91d5-4bd3-9a81-a24fccd7c8a3-etc-kubernetes\") pod \"multus-nwthp\" (UID: \"d108f652-91d5-4bd3-9a81-a24fccd7c8a3\") " pod="openshift-multus/multus-nwthp"
Jul 21 12:39:49.164683 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.164370    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b58bf088-1be7-436e-af10-a747c9f968fb-etc-modprobe-d\") pod \"tuned-4cq9n\" (UID: \"b58bf088-1be7-436e-af10-a747c9f968fb\") " pod="openshift-cluster-node-tuning-operator/tuned-4cq9n"
Jul 21 12:39:49.164683 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.164373    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b58bf088-1be7-436e-af10-a747c9f968fb-etc-kubernetes\") pod \"tuned-4cq9n\" (UID: \"b58bf088-1be7-436e-af10-a747c9f968fb\") " pod="openshift-cluster-node-tuning-operator/tuned-4cq9n"
Jul 21 12:39:49.164683 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.164397    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mbfzx\" (UniqueName: \"kubernetes.io/projected/37f02e4b-2a9f-4e44-8c74-d2428bd97ee5-kube-api-access-mbfzx\") pod \"network-check-target-clxmp\" (UID: \"37f02e4b-2a9f-4e44-8c74-d2428bd97ee5\") " pod="openshift-network-diagnostics/network-check-target-clxmp"
Jul 21 12:39:49.164683 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.164435    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c2aa47a3-b6ce-4807-8e97-aaf79dc1d941-cnibin\") pod \"multus-additional-cni-plugins-vbvlg\" (UID: \"c2aa47a3-b6ce-4807-8e97-aaf79dc1d941\") " pod="openshift-multus/multus-additional-cni-plugins-vbvlg"
Jul 21 12:39:49.164683 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.164485    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b58bf088-1be7-436e-af10-a747c9f968fb-etc-modprobe-d\") pod \"tuned-4cq9n\" (UID: \"b58bf088-1be7-436e-af10-a747c9f968fb\") " pod="openshift-cluster-node-tuning-operator/tuned-4cq9n"
Jul 21 12:39:49.164683 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.164503    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c2aa47a3-b6ce-4807-8e97-aaf79dc1d941-tuning-conf-dir\") pod \"multus-additional-cni-plugins-vbvlg\" (UID: \"c2aa47a3-b6ce-4807-8e97-aaf79dc1d941\") " pod="openshift-multus/multus-additional-cni-plugins-vbvlg"
Jul 21 12:39:49.165538 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.164541    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d71014d4-ae0b-437e-adca-886a8ff5ac7f-original-pull-secret\") pod \"global-pull-secret-syncer-mchwp\" (UID: \"d71014d4-ae0b-437e-adca-886a8ff5ac7f\") " pod="kube-system/global-pull-secret-syncer-mchwp"
Jul 21 12:39:49.165538 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.164593    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d108f652-91d5-4bd3-9a81-a24fccd7c8a3-multus-cni-dir\") pod \"multus-nwthp\" (UID: \"d108f652-91d5-4bd3-9a81-a24fccd7c8a3\") " pod="openshift-multus/multus-nwthp"
Jul 21 12:39:49.165538 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.164616    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/2c5a6a09-909c-4e4c-a05c-7d8f962bb20e-ovnkube-script-lib\") pod \"ovnkube-node-488zb\" (UID: \"2c5a6a09-909c-4e4c-a05c-7d8f962bb20e\") " pod="openshift-ovn-kubernetes/ovnkube-node-488zb"
Jul 21 12:39:49.165538 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.164637    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b58bf088-1be7-436e-af10-a747c9f968fb-lib-modules\") pod \"tuned-4cq9n\" (UID: \"b58bf088-1be7-436e-af10-a747c9f968fb\") " pod="openshift-cluster-node-tuning-operator/tuned-4cq9n"
Jul 21 12:39:49.165538 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.164653    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c2aa47a3-b6ce-4807-8e97-aaf79dc1d941-cni-binary-copy\") pod \"multus-additional-cni-plugins-vbvlg\" (UID: \"c2aa47a3-b6ce-4807-8e97-aaf79dc1d941\") " pod="openshift-multus/multus-additional-cni-plugins-vbvlg"
Jul 21 12:39:49.165538 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.164682    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d108f652-91d5-4bd3-9a81-a24fccd7c8a3-host-var-lib-cni-bin\") pod \"multus-nwthp\" (UID: \"d108f652-91d5-4bd3-9a81-a24fccd7c8a3\") " pod="openshift-multus/multus-nwthp"
Jul 21 12:39:49.165538 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.164701    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b58bf088-1be7-436e-af10-a747c9f968fb-etc-sysconfig\") pod \"tuned-4cq9n\" (UID: \"b58bf088-1be7-436e-af10-a747c9f968fb\") " pod="openshift-cluster-node-tuning-operator/tuned-4cq9n"
Jul 21 12:39:49.165538 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.164718    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b58bf088-1be7-436e-af10-a747c9f968fb-sys\") pod \"tuned-4cq9n\" (UID: \"b58bf088-1be7-436e-af10-a747c9f968fb\") " pod="openshift-cluster-node-tuning-operator/tuned-4cq9n"
Jul 21 12:39:49.165538 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.164734    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/38932224-eb5c-47f9-8515-8d4ba212808d-device-dir\") pod \"aws-ebs-csi-driver-node-xq6pc\" (UID: \"38932224-eb5c-47f9-8515-8d4ba212808d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xq6pc"
Jul 21 12:39:49.165538 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.164754    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/38932224-eb5c-47f9-8515-8d4ba212808d-sys-fs\") pod \"aws-ebs-csi-driver-node-xq6pc\" (UID: \"38932224-eb5c-47f9-8515-8d4ba212808d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xq6pc"
Jul 21 12:39:49.165538 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.164782    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q4c8r\" (UniqueName: \"kubernetes.io/projected/c2aa47a3-b6ce-4807-8e97-aaf79dc1d941-kube-api-access-q4c8r\") pod \"multus-additional-cni-plugins-vbvlg\" (UID: \"c2aa47a3-b6ce-4807-8e97-aaf79dc1d941\") " pod="openshift-multus/multus-additional-cni-plugins-vbvlg"
Jul 21 12:39:49.165538 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.164801    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b58bf088-1be7-436e-af10-a747c9f968fb-etc-sysconfig\") pod \"tuned-4cq9n\" (UID: \"b58bf088-1be7-436e-af10-a747c9f968fb\") " pod="openshift-cluster-node-tuning-operator/tuned-4cq9n"
Jul 21 12:39:49.165538 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.164801    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e5faf5b4-5f7e-44dd-9503-3cc15f07533e-hosts-file\") pod \"node-resolver-z24v8\" (UID: \"e5faf5b4-5f7e-44dd-9503-3cc15f07533e\") " pod="openshift-dns/node-resolver-z24v8"
Jul 21 12:39:49.165538 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.164832    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/38932224-eb5c-47f9-8515-8d4ba212808d-device-dir\") pod \"aws-ebs-csi-driver-node-xq6pc\" (UID: \"38932224-eb5c-47f9-8515-8d4ba212808d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xq6pc"
Jul 21 12:39:49.165538 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.164845    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/d108f652-91d5-4bd3-9a81-a24fccd7c8a3-host-run-k8s-cni-cncf-io\") pod \"multus-nwthp\" (UID: \"d108f652-91d5-4bd3-9a81-a24fccd7c8a3\") " pod="openshift-multus/multus-nwthp"
Jul 21 12:39:49.165538 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.164782    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b58bf088-1be7-436e-af10-a747c9f968fb-lib-modules\") pod \"tuned-4cq9n\" (UID: \"b58bf088-1be7-436e-af10-a747c9f968fb\") " pod="openshift-cluster-node-tuning-operator/tuned-4cq9n"
Jul 21 12:39:49.165538 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.164867    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/38932224-eb5c-47f9-8515-8d4ba212808d-etc-selinux\") pod \"aws-ebs-csi-driver-node-xq6pc\" (UID: \"38932224-eb5c-47f9-8515-8d4ba212808d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xq6pc"
Jul 21 12:39:49.166345 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.164884    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2c5a6a09-909c-4e4c-a05c-7d8f962bb20e-host-run-ovn-kubernetes\") pod \"ovnkube-node-488zb\" (UID: \"2c5a6a09-909c-4e4c-a05c-7d8f962bb20e\") " pod="openshift-ovn-kubernetes/ovnkube-node-488zb"
Jul 21 12:39:49.166345 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.164892    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b58bf088-1be7-436e-af10-a747c9f968fb-sys\") pod \"tuned-4cq9n\" (UID: \"b58bf088-1be7-436e-af10-a747c9f968fb\") " pod="openshift-cluster-node-tuning-operator/tuned-4cq9n"
Jul 21 12:39:49.166345 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.164900    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/543737a1-6fd3-489b-a2b2-9822825c0007-metrics-certs\") pod \"network-metrics-daemon-jfqn8\" (UID: \"543737a1-6fd3-489b-a2b2-9822825c0007\") " pod="openshift-multus/network-metrics-daemon-jfqn8"
Jul 21 12:39:49.166345 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.164849    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/38932224-eb5c-47f9-8515-8d4ba212808d-sys-fs\") pod \"aws-ebs-csi-driver-node-xq6pc\" (UID: \"38932224-eb5c-47f9-8515-8d4ba212808d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xq6pc"
Jul 21 12:39:49.166345 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.164918    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c2aa47a3-b6ce-4807-8e97-aaf79dc1d941-system-cni-dir\") pod \"multus-additional-cni-plugins-vbvlg\" (UID: \"c2aa47a3-b6ce-4807-8e97-aaf79dc1d941\") " pod="openshift-multus/multus-additional-cni-plugins-vbvlg"
Jul 21 12:39:49.166345 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.164932    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/38932224-eb5c-47f9-8515-8d4ba212808d-etc-selinux\") pod \"aws-ebs-csi-driver-node-xq6pc\" (UID: \"38932224-eb5c-47f9-8515-8d4ba212808d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xq6pc"
Jul 21 12:39:49.166345 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.164947    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/2c5a6a09-909c-4e4c-a05c-7d8f962bb20e-run-systemd\") pod \"ovnkube-node-488zb\" (UID: \"2c5a6a09-909c-4e4c-a05c-7d8f962bb20e\") " pod="openshift-ovn-kubernetes/ovnkube-node-488zb"
Jul 21 12:39:49.166345 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.164966    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6z7sn\" (UniqueName: \"kubernetes.io/projected/5adf7abd-fea9-4a2a-8f96-a73bd0a3404a-kube-api-access-6z7sn\") pod \"node-ca-76wr9\" (UID: \"5adf7abd-fea9-4a2a-8f96-a73bd0a3404a\") " pod="openshift-image-registry/node-ca-76wr9"
Jul 21 12:39:49.166345 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.165003    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d71014d4-ae0b-437e-adca-886a8ff5ac7f-dbus\") pod \"global-pull-secret-syncer-mchwp\" (UID: \"d71014d4-ae0b-437e-adca-886a8ff5ac7f\") " pod="kube-system/global-pull-secret-syncer-mchwp"
Jul 21 12:39:49.166345 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.165029    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/d108f652-91d5-4bd3-9a81-a24fccd7c8a3-host-var-lib-cni-multus\") pod \"multus-nwthp\" (UID: \"d108f652-91d5-4bd3-9a81-a24fccd7c8a3\") " pod="openshift-multus/multus-nwthp"
Jul 21 12:39:49.166345 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.165067    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/2b732ef7-474f-4e5e-92b8-03ad2677f408-iptables-alerter-script\") pod \"iptables-alerter-h6bkx\" (UID: \"2b732ef7-474f-4e5e-92b8-03ad2677f408\") " pod="openshift-network-operator/iptables-alerter-h6bkx"
Jul 21 12:39:49.166345 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.165086    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2b732ef7-474f-4e5e-92b8-03ad2677f408-host-slash\") pod \"iptables-alerter-h6bkx\" (UID: \"2b732ef7-474f-4e5e-92b8-03ad2677f408\") " pod="openshift-network-operator/iptables-alerter-h6bkx"
Jul 21 12:39:49.166345 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.165106    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2c5a6a09-909c-4e4c-a05c-7d8f962bb20e-etc-openvswitch\") pod \"ovnkube-node-488zb\" (UID: \"2c5a6a09-909c-4e4c-a05c-7d8f962bb20e\") " pod="openshift-ovn-kubernetes/ovnkube-node-488zb"
Jul 21 12:39:49.166345 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.165122    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2c5a6a09-909c-4e4c-a05c-7d8f962bb20e-host-cni-bin\") pod \"ovnkube-node-488zb\" (UID: \"2c5a6a09-909c-4e4c-a05c-7d8f962bb20e\") " pod="openshift-ovn-kubernetes/ovnkube-node-488zb"
Jul 21 12:39:49.166345 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.165156    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/d108f652-91d5-4bd3-9a81-a24fccd7c8a3-multus-socket-dir-parent\") pod \"multus-nwthp\" (UID: \"d108f652-91d5-4bd3-9a81-a24fccd7c8a3\") " pod="openshift-multus/multus-nwthp"
Jul 21 12:39:49.166345 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.165200    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b58bf088-1be7-436e-af10-a747c9f968fb-etc-sysctl-conf\") pod \"tuned-4cq9n\" (UID: \"b58bf088-1be7-436e-af10-a747c9f968fb\") " pod="openshift-cluster-node-tuning-operator/tuned-4cq9n"
Jul 21 12:39:49.166345 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.165238    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b58bf088-1be7-436e-af10-a747c9f968fb-run\") pod \"tuned-4cq9n\" (UID: \"b58bf088-1be7-436e-af10-a747c9f968fb\") " pod="openshift-cluster-node-tuning-operator/tuned-4cq9n"
Jul 21 12:39:49.166959 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.165283    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2c5a6a09-909c-4e4c-a05c-7d8f962bb20e-var-lib-openvswitch\") pod \"ovnkube-node-488zb\" (UID: \"2c5a6a09-909c-4e4c-a05c-7d8f962bb20e\") " pod="openshift-ovn-kubernetes/ovnkube-node-488zb"
Jul 21 12:39:49.166959 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.165304    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b58bf088-1be7-436e-af10-a747c9f968fb-etc-sysctl-conf\") pod \"tuned-4cq9n\" (UID: \"b58bf088-1be7-436e-af10-a747c9f968fb\") " pod="openshift-cluster-node-tuning-operator/tuned-4cq9n"
Jul 21 12:39:49.166959 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.165311    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/2c5a6a09-909c-4e4c-a05c-7d8f962bb20e-ovnkube-config\") pod \"ovnkube-node-488zb\" (UID: \"2c5a6a09-909c-4e4c-a05c-7d8f962bb20e\") " pod="openshift-ovn-kubernetes/ovnkube-node-488zb"
Jul 21 12:39:49.166959 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.165348    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/2c5a6a09-909c-4e4c-a05c-7d8f962bb20e-ovn-node-metrics-cert\") pod \"ovnkube-node-488zb\" (UID: \"2c5a6a09-909c-4e4c-a05c-7d8f962bb20e\") " pod="openshift-ovn-kubernetes/ovnkube-node-488zb"
Jul 21 12:39:49.166959 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.165366    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b58bf088-1be7-436e-af10-a747c9f968fb-run\") pod \"tuned-4cq9n\" (UID: \"b58bf088-1be7-436e-af10-a747c9f968fb\") " pod="openshift-cluster-node-tuning-operator/tuned-4cq9n"
Jul 21 12:39:49.166959 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.165379    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ccpd7\" (UniqueName: \"kubernetes.io/projected/e5faf5b4-5f7e-44dd-9503-3cc15f07533e-kube-api-access-ccpd7\") pod \"node-resolver-z24v8\" (UID: \"e5faf5b4-5f7e-44dd-9503-3cc15f07533e\") " pod="openshift-dns/node-resolver-z24v8"
Jul 21 12:39:49.166959 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.165408    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/d108f652-91d5-4bd3-9a81-a24fccd7c8a3-multus-daemon-config\") pod \"multus-nwthp\" (UID: \"d108f652-91d5-4bd3-9a81-a24fccd7c8a3\") " pod="openshift-multus/multus-nwthp"
Jul 21 12:39:49.166959 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.165436    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b58bf088-1be7-436e-af10-a747c9f968fb-etc-tuned\") pod \"tuned-4cq9n\" (UID: \"b58bf088-1be7-436e-af10-a747c9f968fb\") " pod="openshift-cluster-node-tuning-operator/tuned-4cq9n"
Jul 21 12:39:49.166959 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.165506    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b58bf088-1be7-436e-af10-a747c9f968fb-tmp\") pod \"tuned-4cq9n\" (UID: \"b58bf088-1be7-436e-af10-a747c9f968fb\") " pod="openshift-cluster-node-tuning-operator/tuned-4cq9n"
Jul 21 12:39:49.166959 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.165739    2556 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 21 12:39:49.169524 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.169403    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b58bf088-1be7-436e-af10-a747c9f968fb-tmp\") pod \"tuned-4cq9n\" (UID: \"b58bf088-1be7-436e-af10-a747c9f968fb\") " pod="openshift-cluster-node-tuning-operator/tuned-4cq9n"
Jul 21 12:39:49.171388 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:39:49.171037    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 21 12:39:49.171388 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:39:49.171059    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 21 12:39:49.171388 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:39:49.171073    2556 projected.go:196] Error preparing data for projected volume kube-api-access-mbfzx for pod openshift-network-diagnostics/network-check-target-clxmp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 12:39:49.171388 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:39:49.171170    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/37f02e4b-2a9f-4e44-8c74-d2428bd97ee5-kube-api-access-mbfzx podName:37f02e4b-2a9f-4e44-8c74-d2428bd97ee5 nodeName:}" failed. No retries permitted until 2026-07-21 12:39:49.671137363 +0000 UTC m=+2.975142431 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-mbfzx" (UniqueName: "kubernetes.io/projected/37f02e4b-2a9f-4e44-8c74-d2428bd97ee5-kube-api-access-mbfzx") pod "network-check-target-clxmp" (UID: "37f02e4b-2a9f-4e44-8c74-d2428bd97ee5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 12:39:49.172011 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.171703    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b58bf088-1be7-436e-af10-a747c9f968fb-etc-tuned\") pod \"tuned-4cq9n\" (UID: \"b58bf088-1be7-436e-af10-a747c9f968fb\") " pod="openshift-cluster-node-tuning-operator/tuned-4cq9n"
Jul 21 12:39:49.174271 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.174168    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g5bnc\" (UniqueName: \"kubernetes.io/projected/38932224-eb5c-47f9-8515-8d4ba212808d-kube-api-access-g5bnc\") pod \"aws-ebs-csi-driver-node-xq6pc\" (UID: \"38932224-eb5c-47f9-8515-8d4ba212808d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xq6pc"
Jul 21 12:39:49.176620 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.176383    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jxlpq\" (UniqueName: \"kubernetes.io/projected/b58bf088-1be7-436e-af10-a747c9f968fb-kube-api-access-jxlpq\") pod \"tuned-4cq9n\" (UID: \"b58bf088-1be7-436e-af10-a747c9f968fb\") " pod="openshift-cluster-node-tuning-operator/tuned-4cq9n"
Jul 21 12:39:49.194984 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.194937    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-2.ec2.internal" event={"ID":"1564417db20202d90def33f3db6f1c9c","Type":"ContainerStarted","Data":"d8125c07d247c88e70c89709995be350b5debdec6c402a9735fbe1230ee0a28c"}
Jul 21 12:39:49.197156 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.197123    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-2.ec2.internal" event={"ID":"88a4b370840ce6eb63d3d3c786c424a4","Type":"ContainerStarted","Data":"9fac6f29392e9ee6e8615e01c66cc871f485ff221a00f39022fc083f171efd16"}
Jul 21 12:39:49.266207 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.266178    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/d108f652-91d5-4bd3-9a81-a24fccd7c8a3-host-var-lib-cni-multus\") pod \"multus-nwthp\" (UID: \"d108f652-91d5-4bd3-9a81-a24fccd7c8a3\") " pod="openshift-multus/multus-nwthp"
Jul 21 12:39:49.266207 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.266209    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/2b732ef7-474f-4e5e-92b8-03ad2677f408-iptables-alerter-script\") pod \"iptables-alerter-h6bkx\" (UID: \"2b732ef7-474f-4e5e-92b8-03ad2677f408\") " pod="openshift-network-operator/iptables-alerter-h6bkx"
Jul 21 12:39:49.266427 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.266229    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2b732ef7-474f-4e5e-92b8-03ad2677f408-host-slash\") pod \"iptables-alerter-h6bkx\" (UID: \"2b732ef7-474f-4e5e-92b8-03ad2677f408\") " pod="openshift-network-operator/iptables-alerter-h6bkx"
Jul 21 12:39:49.266427 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.266247    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2c5a6a09-909c-4e4c-a05c-7d8f962bb20e-etc-openvswitch\") pod \"ovnkube-node-488zb\" (UID: \"2c5a6a09-909c-4e4c-a05c-7d8f962bb20e\") " pod="openshift-ovn-kubernetes/ovnkube-node-488zb"
Jul 21 12:39:49.266427 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.266271    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2c5a6a09-909c-4e4c-a05c-7d8f962bb20e-host-cni-bin\") pod \"ovnkube-node-488zb\" (UID: \"2c5a6a09-909c-4e4c-a05c-7d8f962bb20e\") " pod="openshift-ovn-kubernetes/ovnkube-node-488zb"
Jul 21 12:39:49.266427 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.266333    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2b732ef7-474f-4e5e-92b8-03ad2677f408-host-slash\") pod \"iptables-alerter-h6bkx\" (UID: \"2b732ef7-474f-4e5e-92b8-03ad2677f408\") " pod="openshift-network-operator/iptables-alerter-h6bkx"
Jul 21 12:39:49.266427 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.266335    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2c5a6a09-909c-4e4c-a05c-7d8f962bb20e-etc-openvswitch\") pod \"ovnkube-node-488zb\" (UID: \"2c5a6a09-909c-4e4c-a05c-7d8f962bb20e\") " pod="openshift-ovn-kubernetes/ovnkube-node-488zb"
Jul 21 12:39:49.266427 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.266378    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/d108f652-91d5-4bd3-9a81-a24fccd7c8a3-host-var-lib-cni-multus\") pod \"multus-nwthp\" (UID: \"d108f652-91d5-4bd3-9a81-a24fccd7c8a3\") " pod="openshift-multus/multus-nwthp"
Jul 21 12:39:49.266427 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.266381    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2c5a6a09-909c-4e4c-a05c-7d8f962bb20e-host-cni-bin\") pod \"ovnkube-node-488zb\" (UID: \"2c5a6a09-909c-4e4c-a05c-7d8f962bb20e\") " pod="openshift-ovn-kubernetes/ovnkube-node-488zb"
Jul 21 12:39:49.266427 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.266405    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/d108f652-91d5-4bd3-9a81-a24fccd7c8a3-multus-socket-dir-parent\") pod \"multus-nwthp\" (UID: \"d108f652-91d5-4bd3-9a81-a24fccd7c8a3\") " pod="openshift-multus/multus-nwthp"
Jul 21 12:39:49.266427 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.266426    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2c5a6a09-909c-4e4c-a05c-7d8f962bb20e-var-lib-openvswitch\") pod \"ovnkube-node-488zb\" (UID: \"2c5a6a09-909c-4e4c-a05c-7d8f962bb20e\") " pod="openshift-ovn-kubernetes/ovnkube-node-488zb"
Jul 21 12:39:49.266936 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.266442    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/2c5a6a09-909c-4e4c-a05c-7d8f962bb20e-ovnkube-config\") pod \"ovnkube-node-488zb\" (UID: \"2c5a6a09-909c-4e4c-a05c-7d8f962bb20e\") " pod="openshift-ovn-kubernetes/ovnkube-node-488zb"
Jul 21 12:39:49.266936 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.266460    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/2c5a6a09-909c-4e4c-a05c-7d8f962bb20e-ovn-node-metrics-cert\") pod \"ovnkube-node-488zb\" (UID: \"2c5a6a09-909c-4e4c-a05c-7d8f962bb20e\") " pod="openshift-ovn-kubernetes/ovnkube-node-488zb"
Jul 21 12:39:49.266936 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.266486    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ccpd7\" (UniqueName: \"kubernetes.io/projected/e5faf5b4-5f7e-44dd-9503-3cc15f07533e-kube-api-access-ccpd7\") pod \"node-resolver-z24v8\" (UID: \"e5faf5b4-5f7e-44dd-9503-3cc15f07533e\") " pod="openshift-dns/node-resolver-z24v8"
Jul 21 12:39:49.266936 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.266496    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2c5a6a09-909c-4e4c-a05c-7d8f962bb20e-var-lib-openvswitch\") pod \"ovnkube-node-488zb\" (UID: \"2c5a6a09-909c-4e4c-a05c-7d8f962bb20e\") " pod="openshift-ovn-kubernetes/ovnkube-node-488zb"
Jul 21 12:39:49.266936 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.266510    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/d108f652-91d5-4bd3-9a81-a24fccd7c8a3-multus-daemon-config\") pod \"multus-nwthp\" (UID: \"d108f652-91d5-4bd3-9a81-a24fccd7c8a3\") " pod="openshift-multus/multus-nwthp"
Jul 21 12:39:49.266936 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.266536    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/2c5a6a09-909c-4e4c-a05c-7d8f962bb20e-run-ovn\") pod \"ovnkube-node-488zb\" (UID: \"2c5a6a09-909c-4e4c-a05c-7d8f962bb20e\") " pod="openshift-ovn-kubernetes/ovnkube-node-488zb"
Jul 21 12:39:49.266936 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.266552    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/d108f652-91d5-4bd3-9a81-a24fccd7c8a3-multus-socket-dir-parent\") pod \"multus-nwthp\" (UID: \"d108f652-91d5-4bd3-9a81-a24fccd7c8a3\") " pod="openshift-multus/multus-nwthp"
Jul 21 12:39:49.266936 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.266564    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/2c5a6a09-909c-4e4c-a05c-7d8f962bb20e-systemd-units\") pod \"ovnkube-node-488zb\" (UID: \"2c5a6a09-909c-4e4c-a05c-7d8f962bb20e\") " pod="openshift-ovn-kubernetes/ovnkube-node-488zb"
Jul 21 12:39:49.266936 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.266588    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2c5a6a09-909c-4e4c-a05c-7d8f962bb20e-host-slash\") pod \"ovnkube-node-488zb\" (UID: \"2c5a6a09-909c-4e4c-a05c-7d8f962bb20e\") " pod="openshift-ovn-kubernetes/ovnkube-node-488zb"
Jul 21 12:39:49.266936 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.266612    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6glhm\" (UniqueName: \"kubernetes.io/projected/543737a1-6fd3-489b-a2b2-9822825c0007-kube-api-access-6glhm\") pod \"network-metrics-daemon-jfqn8\" (UID: \"543737a1-6fd3-489b-a2b2-9822825c0007\") " pod="openshift-multus/network-metrics-daemon-jfqn8"
Jul 21 12:39:49.266936 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.266635    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/0f25175e-7fc8-497f-88f1-42b3d583bce2-konnectivity-ca\") pod \"konnectivity-agent-tpxr6\" (UID: \"0f25175e-7fc8-497f-88f1-42b3d583bce2\") " pod="kube-system/konnectivity-agent-tpxr6"
Jul 21 12:39:49.266936 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.266657    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e5faf5b4-5f7e-44dd-9503-3cc15f07533e-tmp-dir\") pod \"node-resolver-z24v8\" (UID: \"e5faf5b4-5f7e-44dd-9503-3cc15f07533e\") " pod="openshift-dns/node-resolver-z24v8"
Jul 21 12:39:49.266936 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.266702    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c2aa47a3-b6ce-4807-8e97-aaf79dc1d941-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-vbvlg\" (UID: \"c2aa47a3-b6ce-4807-8e97-aaf79dc1d941\") " pod="openshift-multus/multus-additional-cni-plugins-vbvlg"
Jul 21 12:39:49.266936 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.266728    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2c5a6a09-909c-4e4c-a05c-7d8f962bb20e-host-run-netns\") pod \"ovnkube-node-488zb\" (UID: \"2c5a6a09-909c-4e4c-a05c-7d8f962bb20e\") " pod="openshift-ovn-kubernetes/ovnkube-node-488zb"
Jul 21 12:39:49.266936 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.266750    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d71014d4-ae0b-437e-adca-886a8ff5ac7f-kubelet-config\") pod \"global-pull-secret-syncer-mchwp\" (UID: \"d71014d4-ae0b-437e-adca-886a8ff5ac7f\") " pod="kube-system/global-pull-secret-syncer-mchwp"
Jul 21 12:39:49.266936 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.266775    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xt6df\" (UniqueName: \"kubernetes.io/projected/d108f652-91d5-4bd3-9a81-a24fccd7c8a3-kube-api-access-xt6df\") pod \"multus-nwthp\" (UID: \"d108f652-91d5-4bd3-9a81-a24fccd7c8a3\") " pod="openshift-multus/multus-nwthp"
Jul 21 12:39:49.266936 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.266804    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-82v6h\" (UniqueName: \"kubernetes.io/projected/2c5a6a09-909c-4e4c-a05c-7d8f962bb20e-kube-api-access-82v6h\") pod \"ovnkube-node-488zb\" (UID: \"2c5a6a09-909c-4e4c-a05c-7d8f962bb20e\") " pod="openshift-ovn-kubernetes/ovnkube-node-488zb"
Jul 21 12:39:49.267733 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.266839    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d108f652-91d5-4bd3-9a81-a24fccd7c8a3-host-run-netns\") pod \"multus-nwthp\" (UID: \"d108f652-91d5-4bd3-9a81-a24fccd7c8a3\") " pod="openshift-multus/multus-nwthp"
Jul 21 12:39:49.267733 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.266866    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c2aa47a3-b6ce-4807-8e97-aaf79dc1d941-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-vbvlg\" (UID: \"c2aa47a3-b6ce-4807-8e97-aaf79dc1d941\") " pod="openshift-multus/multus-additional-cni-plugins-vbvlg"
Jul 21 12:39:49.267733 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.266890    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/2c5a6a09-909c-4e4c-a05c-7d8f962bb20e-host-kubelet\") pod \"ovnkube-node-488zb\" (UID: \"2c5a6a09-909c-4e4c-a05c-7d8f962bb20e\") " pod="openshift-ovn-kubernetes/ovnkube-node-488zb"
Jul 21 12:39:49.267733 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.266914    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2c5a6a09-909c-4e4c-a05c-7d8f962bb20e-run-openvswitch\") pod \"ovnkube-node-488zb\" (UID: \"2c5a6a09-909c-4e4c-a05c-7d8f962bb20e\") " pod="openshift-ovn-kubernetes/ovnkube-node-488zb"
Jul 21 12:39:49.267733 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.266944    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/2c5a6a09-909c-4e4c-a05c-7d8f962bb20e-node-log\") pod \"ovnkube-node-488zb\" (UID: \"2c5a6a09-909c-4e4c-a05c-7d8f962bb20e\") " pod="openshift-ovn-kubernetes/ovnkube-node-488zb"
Jul 21 12:39:49.267733 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.266968    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5adf7abd-fea9-4a2a-8f96-a73bd0a3404a-serviceca\") pod \"node-ca-76wr9\" (UID: \"5adf7abd-fea9-4a2a-8f96-a73bd0a3404a\") " pod="openshift-image-registry/node-ca-76wr9"
Jul 21 12:39:49.267733 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.266995    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d108f652-91d5-4bd3-9a81-a24fccd7c8a3-multus-conf-dir\") pod \"multus-nwthp\" (UID: \"d108f652-91d5-4bd3-9a81-a24fccd7c8a3\") " pod="openshift-multus/multus-nwthp"
Jul 21 12:39:49.267733 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.267032    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hkq64\" (UniqueName: \"kubernetes.io/projected/2b732ef7-474f-4e5e-92b8-03ad2677f408-kube-api-access-hkq64\") pod \"iptables-alerter-h6bkx\" (UID: \"2b732ef7-474f-4e5e-92b8-03ad2677f408\") " pod="openshift-network-operator/iptables-alerter-h6bkx"
Jul 21 12:39:49.267733 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.267062    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c2aa47a3-b6ce-4807-8e97-aaf79dc1d941-os-release\") pod \"multus-additional-cni-plugins-vbvlg\" (UID: \"c2aa47a3-b6ce-4807-8e97-aaf79dc1d941\") " pod="openshift-multus/multus-additional-cni-plugins-vbvlg"
Jul 21 12:39:49.267733 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.267088    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/2c5a6a09-909c-4e4c-a05c-7d8f962bb20e-log-socket\") pod \"ovnkube-node-488zb\" (UID: \"2c5a6a09-909c-4e4c-a05c-7d8f962bb20e\") " pod="openshift-ovn-kubernetes/ovnkube-node-488zb"
Jul 21 12:39:49.267733 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.266912    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/2b732ef7-474f-4e5e-92b8-03ad2677f408-iptables-alerter-script\") pod \"iptables-alerter-h6bkx\" (UID: \"2b732ef7-474f-4e5e-92b8-03ad2677f408\") " pod="openshift-network-operator/iptables-alerter-h6bkx"
Jul 21 12:39:49.267733 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.267130    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/2c5a6a09-909c-4e4c-a05c-7d8f962bb20e-host-cni-netd\") pod \"ovnkube-node-488zb\" (UID: \"2c5a6a09-909c-4e4c-a05c-7d8f962bb20e\") " pod="openshift-ovn-kubernetes/ovnkube-node-488zb"
Jul 21 12:39:49.267733 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.267156    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/2c5a6a09-909c-4e4c-a05c-7d8f962bb20e-env-overrides\") pod \"ovnkube-node-488zb\" (UID: \"2c5a6a09-909c-4e4c-a05c-7d8f962bb20e\") " pod="openshift-ovn-kubernetes/ovnkube-node-488zb"
Jul 21 12:39:49.267733 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.267179    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5adf7abd-fea9-4a2a-8f96-a73bd0a3404a-host\") pod \"node-ca-76wr9\" (UID: \"5adf7abd-fea9-4a2a-8f96-a73bd0a3404a\") " pod="openshift-image-registry/node-ca-76wr9"
Jul 21 12:39:49.267733 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.267205    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/d108f652-91d5-4bd3-9a81-a24fccd7c8a3-hostroot\") pod \"multus-nwthp\" (UID: \"d108f652-91d5-4bd3-9a81-a24fccd7c8a3\") " pod="openshift-multus/multus-nwthp"
Jul 21 12:39:49.267733 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.267230    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/2c5a6a09-909c-4e4c-a05c-7d8f962bb20e-run-ovn\") pod \"ovnkube-node-488zb\" (UID: \"2c5a6a09-909c-4e4c-a05c-7d8f962bb20e\") " pod="openshift-ovn-kubernetes/ovnkube-node-488zb"
Jul 21 12:39:49.267733 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.267242    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/d108f652-91d5-4bd3-9a81-a24fccd7c8a3-host-run-multus-certs\") pod \"multus-nwthp\" (UID: \"d108f652-91d5-4bd3-9a81-a24fccd7c8a3\") " pod="openshift-multus/multus-nwthp"
Jul 21 12:39:49.267733 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.267289    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/d108f652-91d5-4bd3-9a81-a24fccd7c8a3-host-run-multus-certs\") pod \"multus-nwthp\" (UID: \"d108f652-91d5-4bd3-9a81-a24fccd7c8a3\") " pod="openshift-multus/multus-nwthp"
Jul 21 12:39:49.268444 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.267298    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2c5a6a09-909c-4e4c-a05c-7d8f962bb20e-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-488zb\" (UID: \"2c5a6a09-909c-4e4c-a05c-7d8f962bb20e\") " pod="openshift-ovn-kubernetes/ovnkube-node-488zb"
Jul 21 12:39:49.268444 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.267330    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d108f652-91d5-4bd3-9a81-a24fccd7c8a3-system-cni-dir\") pod \"multus-nwthp\" (UID: \"d108f652-91d5-4bd3-9a81-a24fccd7c8a3\") " pod="openshift-multus/multus-nwthp"
Jul 21 12:39:49.268444 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.267350    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/2c5a6a09-909c-4e4c-a05c-7d8f962bb20e-log-socket\") pod \"ovnkube-node-488zb\" (UID: \"2c5a6a09-909c-4e4c-a05c-7d8f962bb20e\") " pod="openshift-ovn-kubernetes/ovnkube-node-488zb"
Jul 21 12:39:49.268444 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.267350    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/2c5a6a09-909c-4e4c-a05c-7d8f962bb20e-ovnkube-config\") pod \"ovnkube-node-488zb\" (UID: \"2c5a6a09-909c-4e4c-a05c-7d8f962bb20e\") " pod="openshift-ovn-kubernetes/ovnkube-node-488zb"
Jul 21 12:39:49.268444 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.267389    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2c5a6a09-909c-4e4c-a05c-7d8f962bb20e-host-slash\") pod \"ovnkube-node-488zb\" (UID: \"2c5a6a09-909c-4e4c-a05c-7d8f962bb20e\") " pod="openshift-ovn-kubernetes/ovnkube-node-488zb"
Jul 21 12:39:49.268444 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.267394    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d108f652-91d5-4bd3-9a81-a24fccd7c8a3-system-cni-dir\") pod \"multus-nwthp\" (UID: \"d108f652-91d5-4bd3-9a81-a24fccd7c8a3\") " pod="openshift-multus/multus-nwthp"
Jul 21 12:39:49.268444 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.267409    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/2c5a6a09-909c-4e4c-a05c-7d8f962bb20e-host-cni-netd\") pod \"ovnkube-node-488zb\" (UID: \"2c5a6a09-909c-4e4c-a05c-7d8f962bb20e\") " pod="openshift-ovn-kubernetes/ovnkube-node-488zb"
Jul 21 12:39:49.268444 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.267463    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c2aa47a3-b6ce-4807-8e97-aaf79dc1d941-os-release\") pod \"multus-additional-cni-plugins-vbvlg\" (UID: \"c2aa47a3-b6ce-4807-8e97-aaf79dc1d941\") " pod="openshift-multus/multus-additional-cni-plugins-vbvlg"
Jul 21 12:39:49.268444 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.267507    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2c5a6a09-909c-4e4c-a05c-7d8f962bb20e-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-488zb\" (UID: \"2c5a6a09-909c-4e4c-a05c-7d8f962bb20e\") " pod="openshift-ovn-kubernetes/ovnkube-node-488zb"
Jul 21 12:39:49.268444 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.267542    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/2c5a6a09-909c-4e4c-a05c-7d8f962bb20e-systemd-units\") pod \"ovnkube-node-488zb\" (UID: \"2c5a6a09-909c-4e4c-a05c-7d8f962bb20e\") " pod="openshift-ovn-kubernetes/ovnkube-node-488zb"
Jul 21 12:39:49.268444 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.267833    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5adf7abd-fea9-4a2a-8f96-a73bd0a3404a-serviceca\") pod \"node-ca-76wr9\" (UID: \"5adf7abd-fea9-4a2a-8f96-a73bd0a3404a\") " pod="openshift-image-registry/node-ca-76wr9"
Jul 21 12:39:49.268444 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.267863    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/2c5a6a09-909c-4e4c-a05c-7d8f962bb20e-env-overrides\") pod \"ovnkube-node-488zb\" (UID: \"2c5a6a09-909c-4e4c-a05c-7d8f962bb20e\") " pod="openshift-ovn-kubernetes/ovnkube-node-488zb"
Jul 21 12:39:49.268444 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.267883    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d108f652-91d5-4bd3-9a81-a24fccd7c8a3-os-release\") pod \"multus-nwthp\" (UID: \"d108f652-91d5-4bd3-9a81-a24fccd7c8a3\") " pod="openshift-multus/multus-nwthp"
Jul 21 12:39:49.268444 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.267888    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/d108f652-91d5-4bd3-9a81-a24fccd7c8a3-multus-daemon-config\") pod \"multus-nwthp\" (UID: \"d108f652-91d5-4bd3-9a81-a24fccd7c8a3\") " pod="openshift-multus/multus-nwthp"
Jul 21 12:39:49.268444 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.267915    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d108f652-91d5-4bd3-9a81-a24fccd7c8a3-cnibin\") pod \"multus-nwthp\" (UID: \"d108f652-91d5-4bd3-9a81-a24fccd7c8a3\") " pod="openshift-multus/multus-nwthp"
Jul 21 12:39:49.268444 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.267926    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5adf7abd-fea9-4a2a-8f96-a73bd0a3404a-host\") pod \"node-ca-76wr9\" (UID: \"5adf7abd-fea9-4a2a-8f96-a73bd0a3404a\") " pod="openshift-image-registry/node-ca-76wr9"
Jul 21 12:39:49.268444 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.267943    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d108f652-91d5-4bd3-9a81-a24fccd7c8a3-cni-binary-copy\") pod \"multus-nwthp\" (UID: \"d108f652-91d5-4bd3-9a81-a24fccd7c8a3\") " pod="openshift-multus/multus-nwthp"
Jul 21 12:39:49.268444 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.267950    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/2c5a6a09-909c-4e4c-a05c-7d8f962bb20e-host-kubelet\") pod \"ovnkube-node-488zb\" (UID: \"2c5a6a09-909c-4e4c-a05c-7d8f962bb20e\") " pod="openshift-ovn-kubernetes/ovnkube-node-488zb"
Jul 21 12:39:49.269583 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.267972    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d108f652-91d5-4bd3-9a81-a24fccd7c8a3-host-var-lib-kubelet\") pod \"multus-nwthp\" (UID: \"d108f652-91d5-4bd3-9a81-a24fccd7c8a3\") " pod="openshift-multus/multus-nwthp"
Jul 21 12:39:49.269583 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.267997    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2c5a6a09-909c-4e4c-a05c-7d8f962bb20e-run-openvswitch\") pod \"ovnkube-node-488zb\" (UID: \"2c5a6a09-909c-4e4c-a05c-7d8f962bb20e\") " pod="openshift-ovn-kubernetes/ovnkube-node-488zb"
Jul 21 12:39:49.269583 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.268002    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/0f25175e-7fc8-497f-88f1-42b3d583bce2-agent-certs\") pod \"konnectivity-agent-tpxr6\" (UID: \"0f25175e-7fc8-497f-88f1-42b3d583bce2\") " pod="kube-system/konnectivity-agent-tpxr6"
Jul 21 12:39:49.269583 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.268005    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/2c5a6a09-909c-4e4c-a05c-7d8f962bb20e-node-log\") pod \"ovnkube-node-488zb\" (UID: \"2c5a6a09-909c-4e4c-a05c-7d8f962bb20e\") " pod="openshift-ovn-kubernetes/ovnkube-node-488zb"
Jul 21 12:39:49.269583 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.268041    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c2aa47a3-b6ce-4807-8e97-aaf79dc1d941-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-vbvlg\" (UID: \"c2aa47a3-b6ce-4807-8e97-aaf79dc1d941\") " pod="openshift-multus/multus-additional-cni-plugins-vbvlg"
Jul 21 12:39:49.269583 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.268050    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/0f25175e-7fc8-497f-88f1-42b3d583bce2-konnectivity-ca\") pod \"konnectivity-agent-tpxr6\" (UID: \"0f25175e-7fc8-497f-88f1-42b3d583bce2\") " pod="kube-system/konnectivity-agent-tpxr6"
Jul 21 12:39:49.269583 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.268062    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d108f652-91d5-4bd3-9a81-a24fccd7c8a3-os-release\") pod \"multus-nwthp\" (UID: \"d108f652-91d5-4bd3-9a81-a24fccd7c8a3\") " pod="openshift-multus/multus-nwthp"
Jul 21 12:39:49.269583 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.268081    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d108f652-91d5-4bd3-9a81-a24fccd7c8a3-multus-conf-dir\") pod \"multus-nwthp\" (UID: \"d108f652-91d5-4bd3-9a81-a24fccd7c8a3\") " pod="openshift-multus/multus-nwthp"
Jul 21 12:39:49.269583 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.268113    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c2aa47a3-b6ce-4807-8e97-aaf79dc1d941-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-vbvlg\" (UID: \"c2aa47a3-b6ce-4807-8e97-aaf79dc1d941\") " pod="openshift-multus/multus-additional-cni-plugins-vbvlg"
Jul 21 12:39:49.269583 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.268121    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d108f652-91d5-4bd3-9a81-a24fccd7c8a3-etc-kubernetes\") pod \"multus-nwthp\" (UID: \"d108f652-91d5-4bd3-9a81-a24fccd7c8a3\") " pod="openshift-multus/multus-nwthp"
Jul 21 12:39:49.269583 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.267973    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/d108f652-91d5-4bd3-9a81-a24fccd7c8a3-hostroot\") pod \"multus-nwthp\" (UID: \"d108f652-91d5-4bd3-9a81-a24fccd7c8a3\") " pod="openshift-multus/multus-nwthp"
Jul 21 12:39:49.269583 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.268186    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d108f652-91d5-4bd3-9a81-a24fccd7c8a3-host-run-netns\") pod \"multus-nwthp\" (UID: \"d108f652-91d5-4bd3-9a81-a24fccd7c8a3\") " pod="openshift-multus/multus-nwthp"
Jul 21 12:39:49.269583 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.268198    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c2aa47a3-b6ce-4807-8e97-aaf79dc1d941-cnibin\") pod \"multus-additional-cni-plugins-vbvlg\" (UID: \"c2aa47a3-b6ce-4807-8e97-aaf79dc1d941\") " pod="openshift-multus/multus-additional-cni-plugins-vbvlg"
Jul 21 12:39:49.269583 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.268238    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d108f652-91d5-4bd3-9a81-a24fccd7c8a3-cnibin\") pod \"multus-nwthp\" (UID: \"d108f652-91d5-4bd3-9a81-a24fccd7c8a3\") " pod="openshift-multus/multus-nwthp"
Jul 21 12:39:49.269583 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.268288    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2c5a6a09-909c-4e4c-a05c-7d8f962bb20e-host-run-netns\") pod \"ovnkube-node-488zb\" (UID: \"2c5a6a09-909c-4e4c-a05c-7d8f962bb20e\") " pod="openshift-ovn-kubernetes/ovnkube-node-488zb"
Jul 21 12:39:49.269583 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.268301    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d108f652-91d5-4bd3-9a81-a24fccd7c8a3-host-var-lib-kubelet\") pod \"multus-nwthp\" (UID: \"d108f652-91d5-4bd3-9a81-a24fccd7c8a3\") " pod="openshift-multus/multus-nwthp"
Jul 21 12:39:49.269583 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.268304    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c2aa47a3-b6ce-4807-8e97-aaf79dc1d941-tuning-conf-dir\") pod \"multus-additional-cni-plugins-vbvlg\" (UID: \"c2aa47a3-b6ce-4807-8e97-aaf79dc1d941\") " pod="openshift-multus/multus-additional-cni-plugins-vbvlg"
Jul 21 12:39:49.269583 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.268325    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c2aa47a3-b6ce-4807-8e97-aaf79dc1d941-cnibin\") pod \"multus-additional-cni-plugins-vbvlg\" (UID: \"c2aa47a3-b6ce-4807-8e97-aaf79dc1d941\") " pod="openshift-multus/multus-additional-cni-plugins-vbvlg"
Jul 21 12:39:49.270544 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.268350    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d71014d4-ae0b-437e-adca-886a8ff5ac7f-kubelet-config\") pod \"global-pull-secret-syncer-mchwp\" (UID: \"d71014d4-ae0b-437e-adca-886a8ff5ac7f\") " pod="kube-system/global-pull-secret-syncer-mchwp"
Jul 21 12:39:49.270544 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.268375    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d71014d4-ae0b-437e-adca-886a8ff5ac7f-original-pull-secret\") pod \"global-pull-secret-syncer-mchwp\" (UID: \"d71014d4-ae0b-437e-adca-886a8ff5ac7f\") " pod="kube-system/global-pull-secret-syncer-mchwp"
Jul 21 12:39:49.270544 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.268403    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d108f652-91d5-4bd3-9a81-a24fccd7c8a3-etc-kubernetes\") pod \"multus-nwthp\" (UID: \"d108f652-91d5-4bd3-9a81-a24fccd7c8a3\") " pod="openshift-multus/multus-nwthp"
Jul 21 12:39:49.270544 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.268438    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d108f652-91d5-4bd3-9a81-a24fccd7c8a3-multus-cni-dir\") pod \"multus-nwthp\" (UID: \"d108f652-91d5-4bd3-9a81-a24fccd7c8a3\") " pod="openshift-multus/multus-nwthp"
Jul 21 12:39:49.270544 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.268438    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c2aa47a3-b6ce-4807-8e97-aaf79dc1d941-tuning-conf-dir\") pod \"multus-additional-cni-plugins-vbvlg\" (UID: \"c2aa47a3-b6ce-4807-8e97-aaf79dc1d941\") " pod="openshift-multus/multus-additional-cni-plugins-vbvlg"
Jul 21 12:39:49.270544 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.268466    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/2c5a6a09-909c-4e4c-a05c-7d8f962bb20e-ovnkube-script-lib\") pod \"ovnkube-node-488zb\" (UID: \"2c5a6a09-909c-4e4c-a05c-7d8f962bb20e\") " pod="openshift-ovn-kubernetes/ovnkube-node-488zb"
Jul 21 12:39:49.270544 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.268519    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c2aa47a3-b6ce-4807-8e97-aaf79dc1d941-cni-binary-copy\") pod \"multus-additional-cni-plugins-vbvlg\" (UID: \"c2aa47a3-b6ce-4807-8e97-aaf79dc1d941\") " pod="openshift-multus/multus-additional-cni-plugins-vbvlg"
Jul 21 12:39:49.270544 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:39:49.268530    2556 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 21 12:39:49.270544 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.268544    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d108f652-91d5-4bd3-9a81-a24fccd7c8a3-cni-binary-copy\") pod \"multus-nwthp\" (UID: \"d108f652-91d5-4bd3-9a81-a24fccd7c8a3\") " pod="openshift-multus/multus-nwthp"
Jul 21 12:39:49.270544 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.268605    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d108f652-91d5-4bd3-9a81-a24fccd7c8a3-host-var-lib-cni-bin\") pod \"multus-nwthp\" (UID: \"d108f652-91d5-4bd3-9a81-a24fccd7c8a3\") " pod="openshift-multus/multus-nwthp"
Jul 21 12:39:49.270544 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.268617    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e5faf5b4-5f7e-44dd-9503-3cc15f07533e-tmp-dir\") pod \"node-resolver-z24v8\" (UID: \"e5faf5b4-5f7e-44dd-9503-3cc15f07533e\") " pod="openshift-dns/node-resolver-z24v8"
Jul 21 12:39:49.270544 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.268650    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d108f652-91d5-4bd3-9a81-a24fccd7c8a3-host-var-lib-cni-bin\") pod \"multus-nwthp\" (UID: \"d108f652-91d5-4bd3-9a81-a24fccd7c8a3\") " pod="openshift-multus/multus-nwthp"
Jul 21 12:39:49.270544 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:39:49.268677    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d71014d4-ae0b-437e-adca-886a8ff5ac7f-original-pull-secret podName:d71014d4-ae0b-437e-adca-886a8ff5ac7f nodeName:}" failed. No retries permitted until 2026-07-21 12:39:49.768658463 +0000 UTC m=+3.072663533 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d71014d4-ae0b-437e-adca-886a8ff5ac7f-original-pull-secret") pod "global-pull-secret-syncer-mchwp" (UID: "d71014d4-ae0b-437e-adca-886a8ff5ac7f") : object "kube-system"/"original-pull-secret" not registered
Jul 21 12:39:49.270544 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.268720    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d108f652-91d5-4bd3-9a81-a24fccd7c8a3-multus-cni-dir\") pod \"multus-nwthp\" (UID: \"d108f652-91d5-4bd3-9a81-a24fccd7c8a3\") " pod="openshift-multus/multus-nwthp"
Jul 21 12:39:49.270544 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.268720    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q4c8r\" (UniqueName: \"kubernetes.io/projected/c2aa47a3-b6ce-4807-8e97-aaf79dc1d941-kube-api-access-q4c8r\") pod \"multus-additional-cni-plugins-vbvlg\" (UID: \"c2aa47a3-b6ce-4807-8e97-aaf79dc1d941\") " pod="openshift-multus/multus-additional-cni-plugins-vbvlg"
Jul 21 12:39:49.270544 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.268756    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e5faf5b4-5f7e-44dd-9503-3cc15f07533e-hosts-file\") pod \"node-resolver-z24v8\" (UID: \"e5faf5b4-5f7e-44dd-9503-3cc15f07533e\") " pod="openshift-dns/node-resolver-z24v8"
Jul 21 12:39:49.270544 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.268777    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/d108f652-91d5-4bd3-9a81-a24fccd7c8a3-host-run-k8s-cni-cncf-io\") pod \"multus-nwthp\" (UID: \"d108f652-91d5-4bd3-9a81-a24fccd7c8a3\") " pod="openshift-multus/multus-nwthp"
Jul 21 12:39:49.271293 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.268795    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2c5a6a09-909c-4e4c-a05c-7d8f962bb20e-host-run-ovn-kubernetes\") pod \"ovnkube-node-488zb\" (UID: \"2c5a6a09-909c-4e4c-a05c-7d8f962bb20e\") " pod="openshift-ovn-kubernetes/ovnkube-node-488zb"
Jul 21 12:39:49.271293 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.268816    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/543737a1-6fd3-489b-a2b2-9822825c0007-metrics-certs\") pod \"network-metrics-daemon-jfqn8\" (UID: \"543737a1-6fd3-489b-a2b2-9822825c0007\") " pod="openshift-multus/network-metrics-daemon-jfqn8"
Jul 21 12:39:49.271293 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.268839    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c2aa47a3-b6ce-4807-8e97-aaf79dc1d941-system-cni-dir\") pod \"multus-additional-cni-plugins-vbvlg\" (UID: \"c2aa47a3-b6ce-4807-8e97-aaf79dc1d941\") " pod="openshift-multus/multus-additional-cni-plugins-vbvlg"
Jul 21 12:39:49.271293 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.268871    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/2c5a6a09-909c-4e4c-a05c-7d8f962bb20e-run-systemd\") pod \"ovnkube-node-488zb\" (UID: \"2c5a6a09-909c-4e4c-a05c-7d8f962bb20e\") " pod="openshift-ovn-kubernetes/ovnkube-node-488zb"
Jul 21 12:39:49.271293 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.268879    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e5faf5b4-5f7e-44dd-9503-3cc15f07533e-hosts-file\") pod \"node-resolver-z24v8\" (UID: \"e5faf5b4-5f7e-44dd-9503-3cc15f07533e\") " pod="openshift-dns/node-resolver-z24v8"
Jul 21 12:39:49.271293 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.268870    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2c5a6a09-909c-4e4c-a05c-7d8f962bb20e-host-run-ovn-kubernetes\") pod \"ovnkube-node-488zb\" (UID: \"2c5a6a09-909c-4e4c-a05c-7d8f962bb20e\") " pod="openshift-ovn-kubernetes/ovnkube-node-488zb"
Jul 21 12:39:49.271293 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.268918    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/d108f652-91d5-4bd3-9a81-a24fccd7c8a3-host-run-k8s-cni-cncf-io\") pod \"multus-nwthp\" (UID: \"d108f652-91d5-4bd3-9a81-a24fccd7c8a3\") " pod="openshift-multus/multus-nwthp"
Jul 21 12:39:49.271293 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.268918    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c2aa47a3-b6ce-4807-8e97-aaf79dc1d941-system-cni-dir\") pod \"multus-additional-cni-plugins-vbvlg\" (UID: \"c2aa47a3-b6ce-4807-8e97-aaf79dc1d941\") " pod="openshift-multus/multus-additional-cni-plugins-vbvlg"
Jul 21 12:39:49.271293 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:39:49.268941    2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 12:39:49.271293 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.268950    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/2c5a6a09-909c-4e4c-a05c-7d8f962bb20e-run-systemd\") pod \"ovnkube-node-488zb\" (UID: \"2c5a6a09-909c-4e4c-a05c-7d8f962bb20e\") " pod="openshift-ovn-kubernetes/ovnkube-node-488zb"
Jul 21 12:39:49.271293 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.268958    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/2c5a6a09-909c-4e4c-a05c-7d8f962bb20e-ovnkube-script-lib\") pod \"ovnkube-node-488zb\" (UID: \"2c5a6a09-909c-4e4c-a05c-7d8f962bb20e\") " pod="openshift-ovn-kubernetes/ovnkube-node-488zb"
Jul 21 12:39:49.271293 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:39:49.268995    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/543737a1-6fd3-489b-a2b2-9822825c0007-metrics-certs podName:543737a1-6fd3-489b-a2b2-9822825c0007 nodeName:}" failed. No retries permitted until 2026-07-21 12:39:49.768981039 +0000 UTC m=+3.072986109 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/543737a1-6fd3-489b-a2b2-9822825c0007-metrics-certs") pod "network-metrics-daemon-jfqn8" (UID: "543737a1-6fd3-489b-a2b2-9822825c0007") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 12:39:49.271293 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.269015    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6z7sn\" (UniqueName: \"kubernetes.io/projected/5adf7abd-fea9-4a2a-8f96-a73bd0a3404a-kube-api-access-6z7sn\") pod \"node-ca-76wr9\" (UID: \"5adf7abd-fea9-4a2a-8f96-a73bd0a3404a\") " pod="openshift-image-registry/node-ca-76wr9"
Jul 21 12:39:49.271293 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.269065    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d71014d4-ae0b-437e-adca-886a8ff5ac7f-dbus\") pod \"global-pull-secret-syncer-mchwp\" (UID: \"d71014d4-ae0b-437e-adca-886a8ff5ac7f\") " pod="kube-system/global-pull-secret-syncer-mchwp"
Jul 21 12:39:49.271293 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.269147    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d71014d4-ae0b-437e-adca-886a8ff5ac7f-dbus\") pod \"global-pull-secret-syncer-mchwp\" (UID: \"d71014d4-ae0b-437e-adca-886a8ff5ac7f\") " pod="kube-system/global-pull-secret-syncer-mchwp"
Jul 21 12:39:49.271293 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.269807    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c2aa47a3-b6ce-4807-8e97-aaf79dc1d941-cni-binary-copy\") pod \"multus-additional-cni-plugins-vbvlg\" (UID: \"c2aa47a3-b6ce-4807-8e97-aaf79dc1d941\") " pod="openshift-multus/multus-additional-cni-plugins-vbvlg"
Jul 21 12:39:49.271293 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.271105    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/2c5a6a09-909c-4e4c-a05c-7d8f962bb20e-ovn-node-metrics-cert\") pod \"ovnkube-node-488zb\" (UID: \"2c5a6a09-909c-4e4c-a05c-7d8f962bb20e\") " pod="openshift-ovn-kubernetes/ovnkube-node-488zb"
Jul 21 12:39:49.271872 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.271365    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/0f25175e-7fc8-497f-88f1-42b3d583bce2-agent-certs\") pod \"konnectivity-agent-tpxr6\" (UID: \"0f25175e-7fc8-497f-88f1-42b3d583bce2\") " pod="kube-system/konnectivity-agent-tpxr6"
Jul 21 12:39:49.277291 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.277258    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xt6df\" (UniqueName: \"kubernetes.io/projected/d108f652-91d5-4bd3-9a81-a24fccd7c8a3-kube-api-access-xt6df\") pod \"multus-nwthp\" (UID: \"d108f652-91d5-4bd3-9a81-a24fccd7c8a3\") " pod="openshift-multus/multus-nwthp"
Jul 21 12:39:49.277729 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.277706    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ccpd7\" (UniqueName: \"kubernetes.io/projected/e5faf5b4-5f7e-44dd-9503-3cc15f07533e-kube-api-access-ccpd7\") pod \"node-resolver-z24v8\" (UID: \"e5faf5b4-5f7e-44dd-9503-3cc15f07533e\") " pod="openshift-dns/node-resolver-z24v8"
Jul 21 12:39:49.282196 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.281581    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6glhm\" (UniqueName: \"kubernetes.io/projected/543737a1-6fd3-489b-a2b2-9822825c0007-kube-api-access-6glhm\") pod \"network-metrics-daemon-jfqn8\" (UID: \"543737a1-6fd3-489b-a2b2-9822825c0007\") " pod="openshift-multus/network-metrics-daemon-jfqn8"
Jul 21 12:39:49.282196 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.281692    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hkq64\" (UniqueName: \"kubernetes.io/projected/2b732ef7-474f-4e5e-92b8-03ad2677f408-kube-api-access-hkq64\") pod \"iptables-alerter-h6bkx\" (UID: \"2b732ef7-474f-4e5e-92b8-03ad2677f408\") " pod="openshift-network-operator/iptables-alerter-h6bkx"
Jul 21 12:39:49.283008 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.282963    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-82v6h\" (UniqueName: \"kubernetes.io/projected/2c5a6a09-909c-4e4c-a05c-7d8f962bb20e-kube-api-access-82v6h\") pod \"ovnkube-node-488zb\" (UID: \"2c5a6a09-909c-4e4c-a05c-7d8f962bb20e\") " pod="openshift-ovn-kubernetes/ovnkube-node-488zb"
Jul 21 12:39:49.283967 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.283944    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6z7sn\" (UniqueName: \"kubernetes.io/projected/5adf7abd-fea9-4a2a-8f96-a73bd0a3404a-kube-api-access-6z7sn\") pod \"node-ca-76wr9\" (UID: \"5adf7abd-fea9-4a2a-8f96-a73bd0a3404a\") " pod="openshift-image-registry/node-ca-76wr9"
Jul 21 12:39:49.284077 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.284053    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q4c8r\" (UniqueName: \"kubernetes.io/projected/c2aa47a3-b6ce-4807-8e97-aaf79dc1d941-kube-api-access-q4c8r\") pod \"multus-additional-cni-plugins-vbvlg\" (UID: \"c2aa47a3-b6ce-4807-8e97-aaf79dc1d941\") " pod="openshift-multus/multus-additional-cni-plugins-vbvlg"
Jul 21 12:39:49.362834 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.362804    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xq6pc"
Jul 21 12:39:49.375815 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:49.375786    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38932224_eb5c_47f9_8515_8d4ba212808d.slice/crio-03c9ecf8e702a4ace74e134f1a9fd60fad3b43e69b82a6d7b9f2b474846ccbbd WatchSource:0}: Error finding container 03c9ecf8e702a4ace74e134f1a9fd60fad3b43e69b82a6d7b9f2b474846ccbbd: Status 404 returned error can't find the container with id 03c9ecf8e702a4ace74e134f1a9fd60fad3b43e69b82a6d7b9f2b474846ccbbd
Jul 21 12:39:49.378103 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.378083    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-4cq9n"
Jul 21 12:39:49.389309 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:49.389282    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb58bf088_1be7_436e_af10_a747c9f968fb.slice/crio-21d652383ecf67d901351eb13e83dae22c0c15c601c570eee7921cf528b3d210 WatchSource:0}: Error finding container 21d652383ecf67d901351eb13e83dae22c0c15c601c570eee7921cf528b3d210: Status 404 returned error can't find the container with id 21d652383ecf67d901351eb13e83dae22c0c15c601c570eee7921cf528b3d210
Jul 21 12:39:49.389921 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.389902    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-z24v8"
Jul 21 12:39:49.397210 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.397189    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-tpxr6"
Jul 21 12:39:49.401268 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:49.401236    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5faf5b4_5f7e_44dd_9503_3cc15f07533e.slice/crio-f3d67593586cef1ada416941d4e8063e21541a70ad933860159fe96eda244865 WatchSource:0}: Error finding container f3d67593586cef1ada416941d4e8063e21541a70ad933860159fe96eda244865: Status 404 returned error can't find the container with id f3d67593586cef1ada416941d4e8063e21541a70ad933860159fe96eda244865
Jul 21 12:39:49.404192 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.404136    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-vbvlg"
Jul 21 12:39:49.408433 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:49.408399    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f25175e_7fc8_497f_88f1_42b3d583bce2.slice/crio-e10e842a6c0604f0f3e99b565e6509b193ba9c2ffe8d4170d3d23263ff888439 WatchSource:0}: Error finding container e10e842a6c0604f0f3e99b565e6509b193ba9c2ffe8d4170d3d23263ff888439: Status 404 returned error can't find the container with id e10e842a6c0604f0f3e99b565e6509b193ba9c2ffe8d4170d3d23263ff888439
Jul 21 12:39:49.413397 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.413376    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-488zb"
Jul 21 12:39:49.414033 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:49.413997    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2aa47a3_b6ce_4807_8e97_aaf79dc1d941.slice/crio-fd4929eb777501945466e23d43f1004642394d63618d0893f8d67cae83d89d6f WatchSource:0}: Error finding container fd4929eb777501945466e23d43f1004642394d63618d0893f8d67cae83d89d6f: Status 404 returned error can't find the container with id fd4929eb777501945466e23d43f1004642394d63618d0893f8d67cae83d89d6f
Jul 21 12:39:49.424840 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:49.424816    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c5a6a09_909c_4e4c_a05c_7d8f962bb20e.slice/crio-e97cc239a944eb70975406a2490f84312bd938dd19340e0690dd728e362835d1 WatchSource:0}: Error finding container e97cc239a944eb70975406a2490f84312bd938dd19340e0690dd728e362835d1: Status 404 returned error can't find the container with id e97cc239a944eb70975406a2490f84312bd938dd19340e0690dd728e362835d1
Jul 21 12:39:49.428838 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.428821    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-76wr9"
Jul 21 12:39:49.437059 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.437037    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-nwthp"
Jul 21 12:39:49.439341 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:49.439309    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5adf7abd_fea9_4a2a_8f96_a73bd0a3404a.slice/crio-a48a6adcd5b7e546375544c0e24d9c549f7d3c9d317a45e3c0eb0e21ff9966c1 WatchSource:0}: Error finding container a48a6adcd5b7e546375544c0e24d9c549f7d3c9d317a45e3c0eb0e21ff9966c1: Status 404 returned error can't find the container with id a48a6adcd5b7e546375544c0e24d9c549f7d3c9d317a45e3c0eb0e21ff9966c1
Jul 21 12:39:49.443332 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.443310    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-h6bkx"
Jul 21 12:39:49.448368 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:49.448342    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd108f652_91d5_4bd3_9a81_a24fccd7c8a3.slice/crio-96adcbc0c7baf8bd78af7a22d9698f3ca3d907762dc56641aeb382f34aad1ddf WatchSource:0}: Error finding container 96adcbc0c7baf8bd78af7a22d9698f3ca3d907762dc56641aeb382f34aad1ddf: Status 404 returned error can't find the container with id 96adcbc0c7baf8bd78af7a22d9698f3ca3d907762dc56641aeb382f34aad1ddf
Jul 21 12:39:49.456810 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:39:49.456787    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b732ef7_474f_4e5e_92b8_03ad2677f408.slice/crio-70daeb938db2d2fb3cf2cc4df9c39d02ddfbc9192780c3ce400ef980f6ff8a28 WatchSource:0}: Error finding container 70daeb938db2d2fb3cf2cc4df9c39d02ddfbc9192780c3ce400ef980f6ff8a28: Status 404 returned error can't find the container with id 70daeb938db2d2fb3cf2cc4df9c39d02ddfbc9192780c3ce400ef980f6ff8a28
Jul 21 12:39:49.503464 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.503442    2556 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 21 12:39:49.671972 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.671889    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mbfzx\" (UniqueName: \"kubernetes.io/projected/37f02e4b-2a9f-4e44-8c74-d2428bd97ee5-kube-api-access-mbfzx\") pod \"network-check-target-clxmp\" (UID: \"37f02e4b-2a9f-4e44-8c74-d2428bd97ee5\") " pod="openshift-network-diagnostics/network-check-target-clxmp"
Jul 21 12:39:49.672135 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:39:49.672113    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 21 12:39:49.672191 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:39:49.672143    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 21 12:39:49.672191 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:39:49.672155    2556 projected.go:196] Error preparing data for projected volume kube-api-access-mbfzx for pod openshift-network-diagnostics/network-check-target-clxmp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 12:39:49.672273 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:39:49.672221    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/37f02e4b-2a9f-4e44-8c74-d2428bd97ee5-kube-api-access-mbfzx podName:37f02e4b-2a9f-4e44-8c74-d2428bd97ee5 nodeName:}" failed. No retries permitted until 2026-07-21 12:39:50.672201841 +0000 UTC m=+3.976206914 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-mbfzx" (UniqueName: "kubernetes.io/projected/37f02e4b-2a9f-4e44-8c74-d2428bd97ee5-kube-api-access-mbfzx") pod "network-check-target-clxmp" (UID: "37f02e4b-2a9f-4e44-8c74-d2428bd97ee5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 12:39:49.772454 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.772410    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d71014d4-ae0b-437e-adca-886a8ff5ac7f-original-pull-secret\") pod \"global-pull-secret-syncer-mchwp\" (UID: \"d71014d4-ae0b-437e-adca-886a8ff5ac7f\") " pod="kube-system/global-pull-secret-syncer-mchwp"
Jul 21 12:39:49.772454 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:49.772458    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/543737a1-6fd3-489b-a2b2-9822825c0007-metrics-certs\") pod \"network-metrics-daemon-jfqn8\" (UID: \"543737a1-6fd3-489b-a2b2-9822825c0007\") " pod="openshift-multus/network-metrics-daemon-jfqn8"
Jul 21 12:39:49.772672 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:39:49.772575    2556 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 21 12:39:49.772672 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:39:49.772584    2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 12:39:49.772672 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:39:49.772642    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/543737a1-6fd3-489b-a2b2-9822825c0007-metrics-certs podName:543737a1-6fd3-489b-a2b2-9822825c0007 nodeName:}" failed. No retries permitted until 2026-07-21 12:39:50.7726282 +0000 UTC m=+4.076633269 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/543737a1-6fd3-489b-a2b2-9822825c0007-metrics-certs") pod "network-metrics-daemon-jfqn8" (UID: "543737a1-6fd3-489b-a2b2-9822825c0007") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 12:39:49.772672 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:39:49.772661    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d71014d4-ae0b-437e-adca-886a8ff5ac7f-original-pull-secret podName:d71014d4-ae0b-437e-adca-886a8ff5ac7f nodeName:}" failed. No retries permitted until 2026-07-21 12:39:50.772650957 +0000 UTC m=+4.076656027 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d71014d4-ae0b-437e-adca-886a8ff5ac7f-original-pull-secret") pod "global-pull-secret-syncer-mchwp" (UID: "d71014d4-ae0b-437e-adca-886a8ff5ac7f") : object "kube-system"/"original-pull-secret" not registered
Jul 21 12:39:50.104610 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:50.104565    2556 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-20 12:34:49 +0000 UTC" deadline="2028-04-18 07:47:10.910591234 +0000 UTC"
Jul 21 12:39:50.104610 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:50.104601    2556 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15283h7m20.805995369s"
Jul 21 12:39:50.200578 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:50.200545    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-h6bkx" event={"ID":"2b732ef7-474f-4e5e-92b8-03ad2677f408","Type":"ContainerStarted","Data":"70daeb938db2d2fb3cf2cc4df9c39d02ddfbc9192780c3ce400ef980f6ff8a28"}
Jul 21 12:39:50.201959 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:50.201929    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-nwthp" event={"ID":"d108f652-91d5-4bd3-9a81-a24fccd7c8a3","Type":"ContainerStarted","Data":"96adcbc0c7baf8bd78af7a22d9698f3ca3d907762dc56641aeb382f34aad1ddf"}
Jul 21 12:39:50.203082 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:50.203054    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-488zb" event={"ID":"2c5a6a09-909c-4e4c-a05c-7d8f962bb20e","Type":"ContainerStarted","Data":"e97cc239a944eb70975406a2490f84312bd938dd19340e0690dd728e362835d1"}
Jul 21 12:39:50.204535 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:50.204508    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-tpxr6" event={"ID":"0f25175e-7fc8-497f-88f1-42b3d583bce2","Type":"ContainerStarted","Data":"e10e842a6c0604f0f3e99b565e6509b193ba9c2ffe8d4170d3d23263ff888439"}
Jul 21 12:39:50.205748 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:50.205712    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-z24v8" event={"ID":"e5faf5b4-5f7e-44dd-9503-3cc15f07533e","Type":"ContainerStarted","Data":"f3d67593586cef1ada416941d4e8063e21541a70ad933860159fe96eda244865"}
Jul 21 12:39:50.206877 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:50.206849    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-76wr9" event={"ID":"5adf7abd-fea9-4a2a-8f96-a73bd0a3404a","Type":"ContainerStarted","Data":"a48a6adcd5b7e546375544c0e24d9c549f7d3c9d317a45e3c0eb0e21ff9966c1"}
Jul 21 12:39:50.208261 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:50.208205    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vbvlg" event={"ID":"c2aa47a3-b6ce-4807-8e97-aaf79dc1d941","Type":"ContainerStarted","Data":"fd4929eb777501945466e23d43f1004642394d63618d0893f8d67cae83d89d6f"}
Jul 21 12:39:50.209465 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:50.209431    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-4cq9n" event={"ID":"b58bf088-1be7-436e-af10-a747c9f968fb","Type":"ContainerStarted","Data":"21d652383ecf67d901351eb13e83dae22c0c15c601c570eee7921cf528b3d210"}
Jul 21 12:39:50.210651 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:50.210628    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xq6pc" event={"ID":"38932224-eb5c-47f9-8515-8d4ba212808d","Type":"ContainerStarted","Data":"03c9ecf8e702a4ace74e134f1a9fd60fad3b43e69b82a6d7b9f2b474846ccbbd"}
Jul 21 12:39:50.684003 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:50.683914    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mbfzx\" (UniqueName: \"kubernetes.io/projected/37f02e4b-2a9f-4e44-8c74-d2428bd97ee5-kube-api-access-mbfzx\") pod \"network-check-target-clxmp\" (UID: \"37f02e4b-2a9f-4e44-8c74-d2428bd97ee5\") " pod="openshift-network-diagnostics/network-check-target-clxmp"
Jul 21 12:39:50.684216 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:39:50.684187    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 21 12:39:50.684216 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:39:50.684214    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 21 12:39:50.684333 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:39:50.684229    2556 projected.go:196] Error preparing data for projected volume kube-api-access-mbfzx for pod openshift-network-diagnostics/network-check-target-clxmp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 12:39:50.684333 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:39:50.684295    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/37f02e4b-2a9f-4e44-8c74-d2428bd97ee5-kube-api-access-mbfzx podName:37f02e4b-2a9f-4e44-8c74-d2428bd97ee5 nodeName:}" failed. No retries permitted until 2026-07-21 12:39:52.684271725 +0000 UTC m=+5.988276799 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-mbfzx" (UniqueName: "kubernetes.io/projected/37f02e4b-2a9f-4e44-8c74-d2428bd97ee5-kube-api-access-mbfzx") pod "network-check-target-clxmp" (UID: "37f02e4b-2a9f-4e44-8c74-d2428bd97ee5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 12:39:50.784918 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:50.784879    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d71014d4-ae0b-437e-adca-886a8ff5ac7f-original-pull-secret\") pod \"global-pull-secret-syncer-mchwp\" (UID: \"d71014d4-ae0b-437e-adca-886a8ff5ac7f\") " pod="kube-system/global-pull-secret-syncer-mchwp"
Jul 21 12:39:50.785137 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:50.784942    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/543737a1-6fd3-489b-a2b2-9822825c0007-metrics-certs\") pod \"network-metrics-daemon-jfqn8\" (UID: \"543737a1-6fd3-489b-a2b2-9822825c0007\") " pod="openshift-multus/network-metrics-daemon-jfqn8"
Jul 21 12:39:50.785137 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:39:50.785091    2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 12:39:50.785270 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:39:50.785160    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/543737a1-6fd3-489b-a2b2-9822825c0007-metrics-certs podName:543737a1-6fd3-489b-a2b2-9822825c0007 nodeName:}" failed. No retries permitted until 2026-07-21 12:39:52.78513788 +0000 UTC m=+6.089142953 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/543737a1-6fd3-489b-a2b2-9822825c0007-metrics-certs") pod "network-metrics-daemon-jfqn8" (UID: "543737a1-6fd3-489b-a2b2-9822825c0007") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 12:39:50.785270 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:39:50.785234    2556 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 21 12:39:50.785386 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:39:50.785273    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d71014d4-ae0b-437e-adca-886a8ff5ac7f-original-pull-secret podName:d71014d4-ae0b-437e-adca-886a8ff5ac7f nodeName:}" failed. No retries permitted until 2026-07-21 12:39:52.785260489 +0000 UTC m=+6.089265562 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d71014d4-ae0b-437e-adca-886a8ff5ac7f-original-pull-secret") pod "global-pull-secret-syncer-mchwp" (UID: "d71014d4-ae0b-437e-adca-886a8ff5ac7f") : object "kube-system"/"original-pull-secret" not registered
Jul 21 12:39:51.105707 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:51.105443    2556 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-20 12:34:49 +0000 UTC" deadline="2028-01-12 09:02:35.574942081 +0000 UTC"
Jul 21 12:39:51.105707 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:51.105676    2556 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12956h22m44.469271158s"
Jul 21 12:39:51.191828 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:51.190892    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jfqn8"
Jul 21 12:39:51.191828 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:39:51.191059    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jfqn8" podUID="543737a1-6fd3-489b-a2b2-9822825c0007"
Jul 21 12:39:51.191828 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:51.191473    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-clxmp"
Jul 21 12:39:51.191828 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:39:51.191569    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-clxmp" podUID="37f02e4b-2a9f-4e44-8c74-d2428bd97ee5"
Jul 21 12:39:51.191828 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:51.191649    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mchwp"
Jul 21 12:39:51.191828 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:39:51.191726    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mchwp" podUID="d71014d4-ae0b-437e-adca-886a8ff5ac7f"
Jul 21 12:39:51.219774 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:51.219005    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-2.ec2.internal" event={"ID":"88a4b370840ce6eb63d3d3c786c424a4","Type":"ContainerStarted","Data":"61128c60ee5ce9591459ea853c3dc92743a297e55644f359db9cc5d7a7dd02cb"}
Jul 21 12:39:52.227228 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:52.227153    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-2.ec2.internal" event={"ID":"1564417db20202d90def33f3db6f1c9c","Type":"ContainerStarted","Data":"c6400bc975dd06a7063ef3827c5ae1321a46b11044007020141703e074211d1c"}
Jul 21 12:39:52.246301 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:52.245651    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-2.ec2.internal" podStartSLOduration=3.245628599 podStartE2EDuration="3.245628599s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 12:39:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:39:51.237822144 +0000 UTC m=+4.541827256" watchObservedRunningTime="2026-07-21 12:39:52.245628599 +0000 UTC m=+5.549633692"
Jul 21 12:39:52.704825 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:52.704110    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mbfzx\" (UniqueName: \"kubernetes.io/projected/37f02e4b-2a9f-4e44-8c74-d2428bd97ee5-kube-api-access-mbfzx\") pod \"network-check-target-clxmp\" (UID: \"37f02e4b-2a9f-4e44-8c74-d2428bd97ee5\") " pod="openshift-network-diagnostics/network-check-target-clxmp"
Jul 21 12:39:52.704825 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:39:52.704311    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 21 12:39:52.704825 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:39:52.704332    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 21 12:39:52.704825 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:39:52.704347    2556 projected.go:196] Error preparing data for projected volume kube-api-access-mbfzx for pod openshift-network-diagnostics/network-check-target-clxmp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 12:39:52.704825 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:39:52.704404    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/37f02e4b-2a9f-4e44-8c74-d2428bd97ee5-kube-api-access-mbfzx podName:37f02e4b-2a9f-4e44-8c74-d2428bd97ee5 nodeName:}" failed. No retries permitted until 2026-07-21 12:39:56.704383289 +0000 UTC m=+10.008388364 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-mbfzx" (UniqueName: "kubernetes.io/projected/37f02e4b-2a9f-4e44-8c74-d2428bd97ee5-kube-api-access-mbfzx") pod "network-check-target-clxmp" (UID: "37f02e4b-2a9f-4e44-8c74-d2428bd97ee5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 12:39:52.804488 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:52.804453    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/543737a1-6fd3-489b-a2b2-9822825c0007-metrics-certs\") pod \"network-metrics-daemon-jfqn8\" (UID: \"543737a1-6fd3-489b-a2b2-9822825c0007\") " pod="openshift-multus/network-metrics-daemon-jfqn8"
Jul 21 12:39:52.804674 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:52.804585    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d71014d4-ae0b-437e-adca-886a8ff5ac7f-original-pull-secret\") pod \"global-pull-secret-syncer-mchwp\" (UID: \"d71014d4-ae0b-437e-adca-886a8ff5ac7f\") " pod="kube-system/global-pull-secret-syncer-mchwp"
Jul 21 12:39:52.805217 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:39:52.804767    2556 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 21 12:39:52.805217 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:39:52.804831    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d71014d4-ae0b-437e-adca-886a8ff5ac7f-original-pull-secret podName:d71014d4-ae0b-437e-adca-886a8ff5ac7f nodeName:}" failed. No retries permitted until 2026-07-21 12:39:56.804812547 +0000 UTC m=+10.108817632 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d71014d4-ae0b-437e-adca-886a8ff5ac7f-original-pull-secret") pod "global-pull-secret-syncer-mchwp" (UID: "d71014d4-ae0b-437e-adca-886a8ff5ac7f") : object "kube-system"/"original-pull-secret" not registered
Jul 21 12:39:52.805217 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:39:52.804863    2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 12:39:52.805217 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:39:52.804928    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/543737a1-6fd3-489b-a2b2-9822825c0007-metrics-certs podName:543737a1-6fd3-489b-a2b2-9822825c0007 nodeName:}" failed. No retries permitted until 2026-07-21 12:39:56.804908439 +0000 UTC m=+10.108913515 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/543737a1-6fd3-489b-a2b2-9822825c0007-metrics-certs") pod "network-metrics-daemon-jfqn8" (UID: "543737a1-6fd3-489b-a2b2-9822825c0007") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 12:39:53.191462 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:53.190653    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mchwp"
Jul 21 12:39:53.191462 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:39:53.190786    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mchwp" podUID="d71014d4-ae0b-437e-adca-886a8ff5ac7f"
Jul 21 12:39:53.191462 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:53.190857    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jfqn8"
Jul 21 12:39:53.191462 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:39:53.190971    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jfqn8" podUID="543737a1-6fd3-489b-a2b2-9822825c0007"
Jul 21 12:39:53.191462 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:53.191017    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-clxmp"
Jul 21 12:39:53.191462 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:39:53.191107    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-clxmp" podUID="37f02e4b-2a9f-4e44-8c74-d2428bd97ee5"
Jul 21 12:39:54.234000 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:54.233890    2556 generic.go:356] "Generic (PLEG): container finished" podID="1564417db20202d90def33f3db6f1c9c" containerID="c6400bc975dd06a7063ef3827c5ae1321a46b11044007020141703e074211d1c" exitCode=0
Jul 21 12:39:54.234000 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:54.233964    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-2.ec2.internal" event={"ID":"1564417db20202d90def33f3db6f1c9c","Type":"ContainerDied","Data":"c6400bc975dd06a7063ef3827c5ae1321a46b11044007020141703e074211d1c"}
Jul 21 12:39:55.188196 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:55.188162    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-clxmp"
Jul 21 12:39:55.188360 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:39:55.188312    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-clxmp" podUID="37f02e4b-2a9f-4e44-8c74-d2428bd97ee5"
Jul 21 12:39:55.188763 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:55.188742    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mchwp"
Jul 21 12:39:55.188870 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:39:55.188849    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mchwp" podUID="d71014d4-ae0b-437e-adca-886a8ff5ac7f"
Jul 21 12:39:55.188929 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:55.188921    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jfqn8"
Jul 21 12:39:55.189029 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:39:55.189011    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jfqn8" podUID="543737a1-6fd3-489b-a2b2-9822825c0007"
Jul 21 12:39:56.741924 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:56.741880    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mbfzx\" (UniqueName: \"kubernetes.io/projected/37f02e4b-2a9f-4e44-8c74-d2428bd97ee5-kube-api-access-mbfzx\") pod \"network-check-target-clxmp\" (UID: \"37f02e4b-2a9f-4e44-8c74-d2428bd97ee5\") " pod="openshift-network-diagnostics/network-check-target-clxmp"
Jul 21 12:39:56.742389 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:39:56.742133    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 21 12:39:56.742389 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:39:56.742156    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 21 12:39:56.742389 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:39:56.742168    2556 projected.go:196] Error preparing data for projected volume kube-api-access-mbfzx for pod openshift-network-diagnostics/network-check-target-clxmp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 12:39:56.742389 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:39:56.742217    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/37f02e4b-2a9f-4e44-8c74-d2428bd97ee5-kube-api-access-mbfzx podName:37f02e4b-2a9f-4e44-8c74-d2428bd97ee5 nodeName:}" failed. No retries permitted until 2026-07-21 12:40:04.742202859 +0000 UTC m=+18.046207930 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-mbfzx" (UniqueName: "kubernetes.io/projected/37f02e4b-2a9f-4e44-8c74-d2428bd97ee5-kube-api-access-mbfzx") pod "network-check-target-clxmp" (UID: "37f02e4b-2a9f-4e44-8c74-d2428bd97ee5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 12:39:56.843169 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:56.843133    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d71014d4-ae0b-437e-adca-886a8ff5ac7f-original-pull-secret\") pod \"global-pull-secret-syncer-mchwp\" (UID: \"d71014d4-ae0b-437e-adca-886a8ff5ac7f\") " pod="kube-system/global-pull-secret-syncer-mchwp"
Jul 21 12:39:56.843315 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:56.843185    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/543737a1-6fd3-489b-a2b2-9822825c0007-metrics-certs\") pod \"network-metrics-daemon-jfqn8\" (UID: \"543737a1-6fd3-489b-a2b2-9822825c0007\") " pod="openshift-multus/network-metrics-daemon-jfqn8"
Jul 21 12:39:56.843364 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:39:56.843321    2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 12:39:56.843396 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:39:56.843372    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/543737a1-6fd3-489b-a2b2-9822825c0007-metrics-certs podName:543737a1-6fd3-489b-a2b2-9822825c0007 nodeName:}" failed. No retries permitted until 2026-07-21 12:40:04.843359427 +0000 UTC m=+18.147364496 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/543737a1-6fd3-489b-a2b2-9822825c0007-metrics-certs") pod "network-metrics-daemon-jfqn8" (UID: "543737a1-6fd3-489b-a2b2-9822825c0007") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 12:39:56.843728 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:39:56.843701    2556 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 21 12:39:56.843854 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:39:56.843753    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d71014d4-ae0b-437e-adca-886a8ff5ac7f-original-pull-secret podName:d71014d4-ae0b-437e-adca-886a8ff5ac7f nodeName:}" failed. No retries permitted until 2026-07-21 12:40:04.843738926 +0000 UTC m=+18.147744000 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d71014d4-ae0b-437e-adca-886a8ff5ac7f-original-pull-secret") pod "global-pull-secret-syncer-mchwp" (UID: "d71014d4-ae0b-437e-adca-886a8ff5ac7f") : object "kube-system"/"original-pull-secret" not registered
Jul 21 12:39:57.189151 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:57.189076    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-clxmp"
Jul 21 12:39:57.189299 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:39:57.189186    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-clxmp" podUID="37f02e4b-2a9f-4e44-8c74-d2428bd97ee5"
Jul 21 12:39:57.189549 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:57.189531    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mchwp"
Jul 21 12:39:57.189656 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:39:57.189634    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mchwp" podUID="d71014d4-ae0b-437e-adca-886a8ff5ac7f"
Jul 21 12:39:57.189717 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:57.189691    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jfqn8"
Jul 21 12:39:57.189799 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:39:57.189777    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jfqn8" podUID="543737a1-6fd3-489b-a2b2-9822825c0007"
Jul 21 12:39:59.191802 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:59.191332    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mchwp"
Jul 21 12:39:59.191802 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:59.191384    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jfqn8"
Jul 21 12:39:59.191802 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:39:59.191384    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-clxmp"
Jul 21 12:39:59.191802 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:39:59.191483    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mchwp" podUID="d71014d4-ae0b-437e-adca-886a8ff5ac7f"
Jul 21 12:39:59.191802 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:39:59.191620    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jfqn8" podUID="543737a1-6fd3-489b-a2b2-9822825c0007"
Jul 21 12:39:59.191802 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:39:59.191694    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-clxmp" podUID="37f02e4b-2a9f-4e44-8c74-d2428bd97ee5"
Jul 21 12:40:01.187876 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:01.187844    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jfqn8"
Jul 21 12:40:01.187876 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:01.187844    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-clxmp"
Jul 21 12:40:01.188370 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:01.188031    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mchwp"
Jul 21 12:40:01.188370 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:40:01.188040    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jfqn8" podUID="543737a1-6fd3-489b-a2b2-9822825c0007"
Jul 21 12:40:01.188370 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:40:01.188129    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-clxmp" podUID="37f02e4b-2a9f-4e44-8c74-d2428bd97ee5"
Jul 21 12:40:01.188370 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:40:01.188204    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mchwp" podUID="d71014d4-ae0b-437e-adca-886a8ff5ac7f"
Jul 21 12:40:03.188205 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:03.188172    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jfqn8"
Jul 21 12:40:03.188654 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:03.188229    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mchwp"
Jul 21 12:40:03.188654 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:03.188172    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-clxmp"
Jul 21 12:40:03.188654 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:40:03.188307    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jfqn8" podUID="543737a1-6fd3-489b-a2b2-9822825c0007"
Jul 21 12:40:03.188654 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:40:03.188363    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-clxmp" podUID="37f02e4b-2a9f-4e44-8c74-d2428bd97ee5"
Jul 21 12:40:03.188654 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:40:03.188437    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mchwp" podUID="d71014d4-ae0b-437e-adca-886a8ff5ac7f"
Jul 21 12:40:04.806787 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:04.806750    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mbfzx\" (UniqueName: \"kubernetes.io/projected/37f02e4b-2a9f-4e44-8c74-d2428bd97ee5-kube-api-access-mbfzx\") pod \"network-check-target-clxmp\" (UID: \"37f02e4b-2a9f-4e44-8c74-d2428bd97ee5\") " pod="openshift-network-diagnostics/network-check-target-clxmp"
Jul 21 12:40:04.807229 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:40:04.806922    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 21 12:40:04.807229 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:40:04.806950    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 21 12:40:04.807229 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:40:04.806965    2556 projected.go:196] Error preparing data for projected volume kube-api-access-mbfzx for pod openshift-network-diagnostics/network-check-target-clxmp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 12:40:04.807229 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:40:04.807050    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/37f02e4b-2a9f-4e44-8c74-d2428bd97ee5-kube-api-access-mbfzx podName:37f02e4b-2a9f-4e44-8c74-d2428bd97ee5 nodeName:}" failed. No retries permitted until 2026-07-21 12:40:20.807027589 +0000 UTC m=+34.111032674 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-mbfzx" (UniqueName: "kubernetes.io/projected/37f02e4b-2a9f-4e44-8c74-d2428bd97ee5-kube-api-access-mbfzx") pod "network-check-target-clxmp" (UID: "37f02e4b-2a9f-4e44-8c74-d2428bd97ee5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 12:40:04.908067 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:04.908033    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d71014d4-ae0b-437e-adca-886a8ff5ac7f-original-pull-secret\") pod \"global-pull-secret-syncer-mchwp\" (UID: \"d71014d4-ae0b-437e-adca-886a8ff5ac7f\") " pod="kube-system/global-pull-secret-syncer-mchwp"
Jul 21 12:40:04.908067 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:04.908070    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/543737a1-6fd3-489b-a2b2-9822825c0007-metrics-certs\") pod \"network-metrics-daemon-jfqn8\" (UID: \"543737a1-6fd3-489b-a2b2-9822825c0007\") " pod="openshift-multus/network-metrics-daemon-jfqn8"
Jul 21 12:40:04.908286 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:40:04.908164    2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 12:40:04.908286 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:40:04.908183    2556 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 21 12:40:04.908286 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:40:04.908214    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/543737a1-6fd3-489b-a2b2-9822825c0007-metrics-certs podName:543737a1-6fd3-489b-a2b2-9822825c0007 nodeName:}" failed. No retries permitted until 2026-07-21 12:40:20.908200366 +0000 UTC m=+34.212205435 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/543737a1-6fd3-489b-a2b2-9822825c0007-metrics-certs") pod "network-metrics-daemon-jfqn8" (UID: "543737a1-6fd3-489b-a2b2-9822825c0007") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 12:40:04.908286 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:40:04.908245    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d71014d4-ae0b-437e-adca-886a8ff5ac7f-original-pull-secret podName:d71014d4-ae0b-437e-adca-886a8ff5ac7f nodeName:}" failed. No retries permitted until 2026-07-21 12:40:20.908225211 +0000 UTC m=+34.212230293 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d71014d4-ae0b-437e-adca-886a8ff5ac7f-original-pull-secret") pod "global-pull-secret-syncer-mchwp" (UID: "d71014d4-ae0b-437e-adca-886a8ff5ac7f") : object "kube-system"/"original-pull-secret" not registered
Jul 21 12:40:05.187878 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:05.187851    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jfqn8"
Jul 21 12:40:05.187878 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:05.187878    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-clxmp"
Jul 21 12:40:05.188053 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:05.187904    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mchwp"
Jul 21 12:40:05.188053 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:40:05.187990    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-clxmp" podUID="37f02e4b-2a9f-4e44-8c74-d2428bd97ee5"
Jul 21 12:40:05.188135 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:40:05.188094    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mchwp" podUID="d71014d4-ae0b-437e-adca-886a8ff5ac7f"
Jul 21 12:40:05.188179 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:40:05.188151    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jfqn8" podUID="543737a1-6fd3-489b-a2b2-9822825c0007"
Jul 21 12:40:07.189286 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:07.189258    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jfqn8"
Jul 21 12:40:07.189719 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:07.189371    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-clxmp"
Jul 21 12:40:07.189719 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:40:07.189407    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jfqn8" podUID="543737a1-6fd3-489b-a2b2-9822825c0007"
Jul 21 12:40:07.189719 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:40:07.189490    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-clxmp" podUID="37f02e4b-2a9f-4e44-8c74-d2428bd97ee5"
Jul 21 12:40:07.189719 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:07.189540    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mchwp"
Jul 21 12:40:07.189719 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:40:07.189660    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mchwp" podUID="d71014d4-ae0b-437e-adca-886a8ff5ac7f"
Jul 21 12:40:09.188083 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:09.188055    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jfqn8"
Jul 21 12:40:09.188412 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:09.188058    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-clxmp"
Jul 21 12:40:09.188412 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:40:09.188195    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jfqn8" podUID="543737a1-6fd3-489b-a2b2-9822825c0007"
Jul 21 12:40:09.188412 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:40:09.188280    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-clxmp" podUID="37f02e4b-2a9f-4e44-8c74-d2428bd97ee5"
Jul 21 12:40:09.188412 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:09.188314    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mchwp"
Jul 21 12:40:09.188805 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:40:09.188778    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mchwp" podUID="d71014d4-ae0b-437e-adca-886a8ff5ac7f"
Jul 21 12:40:10.273362 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:10.272957    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-2.ec2.internal" event={"ID":"1564417db20202d90def33f3db6f1c9c","Type":"ContainerStarted","Data":"2e20ca423f7a4ac7d9f751a75db94d3400e1337ea88d25fa9957bce22753d064"}
Jul 21 12:40:10.275439 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:10.275410    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-nwthp" event={"ID":"d108f652-91d5-4bd3-9a81-a24fccd7c8a3","Type":"ContainerStarted","Data":"6477d5ab1d234de603b331868ace3140e924568e124c2a26ecb25e342f12232f"}
Jul 21 12:40:10.279077 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:10.279053    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-488zb" event={"ID":"2c5a6a09-909c-4e4c-a05c-7d8f962bb20e","Type":"ContainerStarted","Data":"22da9cf4072a44e75a4177d007fdb4a9afb377a2e4105d704449b3879274f9a9"}
Jul 21 12:40:10.279218 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:10.279133    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-488zb" event={"ID":"2c5a6a09-909c-4e4c-a05c-7d8f962bb20e","Type":"ContainerStarted","Data":"a26c0c0906015f862ca6f77f754fac403f1f4143fc82638bbc4e27d2832fe7d2"}
Jul 21 12:40:10.279218 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:10.279182    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-488zb" event={"ID":"2c5a6a09-909c-4e4c-a05c-7d8f962bb20e","Type":"ContainerStarted","Data":"4dd1046501d9ea6dd21501b941d9b97ac24210568f0b14d0a962babc96c6cb2e"}
Jul 21 12:40:10.279327 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:10.279228    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-488zb" event={"ID":"2c5a6a09-909c-4e4c-a05c-7d8f962bb20e","Type":"ContainerStarted","Data":"8c7d6cc50967485182e0ec0ea314b75153b54607f24f4f4f6d456fc92e00ff5a"}
Jul 21 12:40:10.279327 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:10.279277    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-488zb" event={"ID":"2c5a6a09-909c-4e4c-a05c-7d8f962bb20e","Type":"ContainerStarted","Data":"369342423ee9ffc17a814c32187d0affb694eecb9d33edbea9a1bf794bb4ff52"}
Jul 21 12:40:10.280190 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:10.280163    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-488zb" event={"ID":"2c5a6a09-909c-4e4c-a05c-7d8f962bb20e","Type":"ContainerStarted","Data":"e9cd8455b95e19b09d0e78549b477220a42d0e6ab4b8b5b491fa370cdae72a6f"}
Jul 21 12:40:10.283908 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:10.283880    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-tpxr6" event={"ID":"0f25175e-7fc8-497f-88f1-42b3d583bce2","Type":"ContainerStarted","Data":"6ad6cd6f4d3c6ac8bd1348b95ab67862c27f43b8ba52b863e29bdcc5883f631f"}
Jul 21 12:40:10.285811 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:10.285791    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-z24v8" event={"ID":"e5faf5b4-5f7e-44dd-9503-3cc15f07533e","Type":"ContainerStarted","Data":"a1836e0e5bc15d229c026f45e00c4372e082c7445de7f3f8113d62abe69296de"}
Jul 21 12:40:10.287451 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:10.287421    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-76wr9" event={"ID":"5adf7abd-fea9-4a2a-8f96-a73bd0a3404a","Type":"ContainerStarted","Data":"cc0fc98c92baea04b7487a424066982df7dbd9ce023389eb6819d37bc5972f34"}
Jul 21 12:40:10.289635 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:10.289108    2556 generic.go:356] "Generic (PLEG): container finished" podID="c2aa47a3-b6ce-4807-8e97-aaf79dc1d941" containerID="43b642d3fbfa1c17d6c958458a62ed0b0c3ba696969fd64079b2f61c28cab4c1" exitCode=0
Jul 21 12:40:10.289635 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:10.289149    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vbvlg" event={"ID":"c2aa47a3-b6ce-4807-8e97-aaf79dc1d941","Type":"ContainerDied","Data":"43b642d3fbfa1c17d6c958458a62ed0b0c3ba696969fd64079b2f61c28cab4c1"}
Jul 21 12:40:10.290944 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:10.290916    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-4cq9n" event={"ID":"b58bf088-1be7-436e-af10-a747c9f968fb","Type":"ContainerStarted","Data":"12a8331304d1a875eb41f2f2b2610af151783bf87b9a20fbdccaa0c6c89b5e18"}
Jul 21 12:40:10.292869 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:10.292846    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xq6pc" event={"ID":"38932224-eb5c-47f9-8515-8d4ba212808d","Type":"ContainerStarted","Data":"137474ff14266dcc86be6a055d3be172ddb0ce8a608a3826d9992377a86bf286"}
Jul 21 12:40:10.293281 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:10.293241    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-2.ec2.internal" podStartSLOduration=21.293225393 podStartE2EDuration="21.293225393s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 12:39:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:40:10.293000697 +0000 UTC m=+23.597005786" watchObservedRunningTime="2026-07-21 12:40:10.293225393 +0000 UTC m=+23.597230484"
Jul 21 12:40:10.334887 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:10.334833    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-tpxr6" podStartSLOduration=3.5254028269999997 podStartE2EDuration="23.334814036s" totalImagesPullingTime="19.809411209s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:39:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:40:10.309819895 +0000 UTC m=+23.613824986" watchObservedRunningTime="2026-07-21 12:40:10.334814036 +0000 UTC m=+23.638819127"
Jul 21 12:40:10.352199 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:10.352161    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-4cq9n" podStartSLOduration=3.524262564 podStartE2EDuration="23.352148341s" totalImagesPullingTime="19.827885777s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 12:39:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:40:10.351595082 +0000 UTC m=+23.655600173" watchObservedRunningTime="2026-07-21 12:40:10.352148341 +0000 UTC m=+23.656153430"
Jul 21 12:40:10.367097 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:10.367056    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-z24v8" podStartSLOduration=6.584683065 podStartE2EDuration="22.367039762s" totalImagesPullingTime="15.782356697s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 12:39:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:40:10.366724617 +0000 UTC m=+23.670729708" watchObservedRunningTime="2026-07-21 12:40:10.367039762 +0000 UTC m=+23.671044852"
Jul 21 12:40:10.381541 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:10.381506    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-76wr9" podStartSLOduration=2.791799716 podStartE2EDuration="22.381494616s" totalImagesPullingTime="19.5896949s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 12:39:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:40:10.381428641 +0000 UTC m=+23.685433731" watchObservedRunningTime="2026-07-21 12:40:10.381494616 +0000 UTC m=+23.685499706"
Jul 21 12:40:10.402429 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:10.402396    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-nwthp" podStartSLOduration=2.56954596 podStartE2EDuration="22.402389888s" totalImagesPullingTime="19.832843928s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 12:39:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:40:10.4023196 +0000 UTC m=+23.706324691" watchObservedRunningTime="2026-07-21 12:40:10.402389888 +0000 UTC m=+23.706394978"
Jul 21 12:40:11.087892 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:11.087871    2556 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 21 12:40:11.145834 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:11.145725    2556 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-21T12:40:11.087887494Z","UUID":"7ba416cd-1ed9-414c-89ca-4d05d7f55908","Handler":null,"Name":"","Endpoint":""}
Jul 21 12:40:11.147277 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:11.147236    2556 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 21 12:40:11.147378 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:11.147290    2556 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 21 12:40:11.192171 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:11.192144    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-clxmp"
Jul 21 12:40:11.192171 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:11.192163    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jfqn8"
Jul 21 12:40:11.192561 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:40:11.192252    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-clxmp" podUID="37f02e4b-2a9f-4e44-8c74-d2428bd97ee5"
Jul 21 12:40:11.192561 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:40:11.192298    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jfqn8" podUID="543737a1-6fd3-489b-a2b2-9822825c0007"
Jul 21 12:40:11.192561 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:11.192343    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mchwp"
Jul 21 12:40:11.192561 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:40:11.192434    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mchwp" podUID="d71014d4-ae0b-437e-adca-886a8ff5ac7f"
Jul 21 12:40:11.297572 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:11.297538    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xq6pc" event={"ID":"38932224-eb5c-47f9-8515-8d4ba212808d","Type":"ContainerStarted","Data":"2bccf0662f701fd8031f15424073d348bf29802764945cd2b5a81e8e0cd0ef45"}
Jul 21 12:40:11.299059 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:11.299004    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-h6bkx" event={"ID":"2b732ef7-474f-4e5e-92b8-03ad2677f408","Type":"ContainerStarted","Data":"b933dee81ae40505901b75d1bd039d14a556258fa70bcb38ca859706c73b3cbb"}
Jul 21 12:40:11.319703 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:11.319663    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-h6bkx" podStartSLOduration=7.592793767 podStartE2EDuration="23.319647427s" totalImagesPullingTime="15.72685366s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 12:39:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:40:11.319077227 +0000 UTC m=+24.623082318" watchObservedRunningTime="2026-07-21 12:40:11.319647427 +0000 UTC m=+24.623652520"
Jul 21 12:40:11.829420 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:11.829383    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-tpxr6"
Jul 21 12:40:11.830008 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:11.829986    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-tpxr6"
Jul 21 12:40:12.305774 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:12.305718    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-488zb" event={"ID":"2c5a6a09-909c-4e4c-a05c-7d8f962bb20e","Type":"ContainerStarted","Data":"425a6abc78a25f3072eb0f2932522e8e3a0c1a2b0dd54211f8fd703870797954"}
Jul 21 12:40:12.306432 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:12.306004    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-tpxr6"
Jul 21 12:40:12.306432 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:12.306400    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-tpxr6"
Jul 21 12:40:13.188662 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:13.188389    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jfqn8"
Jul 21 12:40:13.188812 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:13.188401    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-clxmp"
Jul 21 12:40:13.188812 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:40:13.188763    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jfqn8" podUID="543737a1-6fd3-489b-a2b2-9822825c0007"
Jul 21 12:40:13.188929 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:13.188453    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mchwp"
Jul 21 12:40:13.188929 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:40:13.188830    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-clxmp" podUID="37f02e4b-2a9f-4e44-8c74-d2428bd97ee5"
Jul 21 12:40:13.189055 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:40:13.188937    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mchwp" podUID="d71014d4-ae0b-437e-adca-886a8ff5ac7f"
Jul 21 12:40:13.311084 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:13.311046    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xq6pc" event={"ID":"38932224-eb5c-47f9-8515-8d4ba212808d","Type":"ContainerStarted","Data":"47ddd3503689403ffeca0b3771d531b46e3d0d829367a72f483b4131e1ffaaa7"}
Jul 21 12:40:13.329641 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:13.329595    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xq6pc" podStartSLOduration=3.286529197 podStartE2EDuration="26.329576187s" totalImagesPullingTime="23.04304699s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 12:39:47 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:40:13.329456826 +0000 UTC m=+26.633461917" watchObservedRunningTime="2026-07-21 12:40:13.329576187 +0000 UTC m=+26.633581281"
Jul 21 12:40:13.414587 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:13.414541    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-tpxr6_0f25175e-7fc8-497f-88f1-42b3d583bce2/konnectivity-agent/0.log" limit=1024
Jul 21 12:40:13.503416 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:13.503305    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-tpxr6_0f25175e-7fc8-497f-88f1-42b3d583bce2/konnectivity-agent/0.log" limit=1024
Jul 21 12:40:15.188605 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:15.188576    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-clxmp"
Jul 21 12:40:15.188605 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:15.188592    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mchwp"
Jul 21 12:40:15.189245 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:40:15.188697    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-clxmp" podUID="37f02e4b-2a9f-4e44-8c74-d2428bd97ee5"
Jul 21 12:40:15.189245 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:15.188751    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jfqn8"
Jul 21 12:40:15.189245 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:40:15.188867    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mchwp" podUID="d71014d4-ae0b-437e-adca-886a8ff5ac7f"
Jul 21 12:40:15.189245 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:40:15.188954    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jfqn8" podUID="543737a1-6fd3-489b-a2b2-9822825c0007"
Jul 21 12:40:15.320424 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:15.320219    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-488zb" event={"ID":"2c5a6a09-909c-4e4c-a05c-7d8f962bb20e","Type":"ContainerStarted","Data":"727d6976b296bc5e896ebcf7f943d3e523a40cd24246b0f7861bb75101e26c78"}
Jul 21 12:40:15.320542 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:15.320495    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-488zb"
Jul 21 12:40:15.320597 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:15.320545    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-488zb"
Jul 21 12:40:15.320597 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:15.320589    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-488zb"
Jul 21 12:40:15.338456 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:15.338361    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-488zb"
Jul 21 12:40:15.338557 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:15.338518    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-488zb"
Jul 21 12:40:15.346984 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:15.346797    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-488zb" podStartSLOduration=8.49362766 podStartE2EDuration="28.346782543s" totalImagesPullingTime="19.853154883s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 12:39:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:40:15.346058386 +0000 UTC m=+28.650063501" watchObservedRunningTime="2026-07-21 12:40:15.346782543 +0000 UTC m=+28.650787633"
Jul 21 12:40:15.528536 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:15.528505    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-tpxr6_0f25175e-7fc8-497f-88f1-42b3d583bce2/konnectivity-agent/0.log" limit=1024
Jul 21 12:40:15.591329 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:15.591235    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-tpxr6_0f25175e-7fc8-497f-88f1-42b3d583bce2/konnectivity-agent/0.log" limit=1024
Jul 21 12:40:15.962635 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:15.962553    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-tpxr6_0f25175e-7fc8-497f-88f1-42b3d583bce2/konnectivity-agent/0.log" limit=1024
Jul 21 12:40:16.018757 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:16.018726    2556 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-tpxr6_0f25175e-7fc8-497f-88f1-42b3d583bce2/konnectivity-agent/0.log" limit=1024
Jul 21 12:40:16.324079 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:16.324045    2556 generic.go:356] "Generic (PLEG): container finished" podID="c2aa47a3-b6ce-4807-8e97-aaf79dc1d941" containerID="97a646bf86fcbb05e3514e6fcf6fb23699e07f2c94b1e745e842a593eddb48cf" exitCode=0
Jul 21 12:40:16.324603 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:16.324079    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vbvlg" event={"ID":"c2aa47a3-b6ce-4807-8e97-aaf79dc1d941","Type":"ContainerDied","Data":"97a646bf86fcbb05e3514e6fcf6fb23699e07f2c94b1e745e842a593eddb48cf"}
Jul 21 12:40:17.190641 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:17.190614    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mchwp"
Jul 21 12:40:17.190776 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:17.190714    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-clxmp"
Jul 21 12:40:17.190776 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:40:17.190730    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mchwp" podUID="d71014d4-ae0b-437e-adca-886a8ff5ac7f"
Jul 21 12:40:17.190776 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:17.190744    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jfqn8"
Jul 21 12:40:17.190957 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:40:17.190831    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-clxmp" podUID="37f02e4b-2a9f-4e44-8c74-d2428bd97ee5"
Jul 21 12:40:17.190957 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:40:17.190924    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jfqn8" podUID="543737a1-6fd3-489b-a2b2-9822825c0007"
Jul 21 12:40:17.420241 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:17.420219    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-clxmp"]
Jul 21 12:40:17.420526 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:17.420304    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-clxmp"
Jul 21 12:40:17.420526 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:40:17.420381    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-clxmp" podUID="37f02e4b-2a9f-4e44-8c74-d2428bd97ee5"
Jul 21 12:40:17.420767 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:17.420746    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-mchwp"]
Jul 21 12:40:17.420831 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:17.420821    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mchwp"
Jul 21 12:40:17.420911 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:40:17.420896    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mchwp" podUID="d71014d4-ae0b-437e-adca-886a8ff5ac7f"
Jul 21 12:40:17.422570 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:17.422548    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-jfqn8"]
Jul 21 12:40:17.422681 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:17.422638    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jfqn8"
Jul 21 12:40:17.422747 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:40:17.422731    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jfqn8" podUID="543737a1-6fd3-489b-a2b2-9822825c0007"
Jul 21 12:40:18.329879 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:18.329851    2556 generic.go:356] "Generic (PLEG): container finished" podID="c2aa47a3-b6ce-4807-8e97-aaf79dc1d941" containerID="9f13cdcd01ba56c344ccff95e3b4b7d807a1a78dc89d5ec829e956d3248771b3" exitCode=0
Jul 21 12:40:18.330036 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:18.329902    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vbvlg" event={"ID":"c2aa47a3-b6ce-4807-8e97-aaf79dc1d941","Type":"ContainerDied","Data":"9f13cdcd01ba56c344ccff95e3b4b7d807a1a78dc89d5ec829e956d3248771b3"}
Jul 21 12:40:19.188338 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:19.188306    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jfqn8"
Jul 21 12:40:19.188338 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:19.188336    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-clxmp"
Jul 21 12:40:19.188722 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:19.188348    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mchwp"
Jul 21 12:40:19.188722 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:40:19.188438    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jfqn8" podUID="543737a1-6fd3-489b-a2b2-9822825c0007"
Jul 21 12:40:19.188722 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:40:19.188512    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-clxmp" podUID="37f02e4b-2a9f-4e44-8c74-d2428bd97ee5"
Jul 21 12:40:19.188722 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:40:19.188605    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mchwp" podUID="d71014d4-ae0b-437e-adca-886a8ff5ac7f"
Jul 21 12:40:20.339104 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:20.339068    2556 generic.go:356] "Generic (PLEG): container finished" podID="c2aa47a3-b6ce-4807-8e97-aaf79dc1d941" containerID="766818caec14a367afa7af0dbda712da5532245633e061793cc5f134ccac625c" exitCode=0
Jul 21 12:40:20.339434 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:20.339115    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vbvlg" event={"ID":"c2aa47a3-b6ce-4807-8e97-aaf79dc1d941","Type":"ContainerDied","Data":"766818caec14a367afa7af0dbda712da5532245633e061793cc5f134ccac625c"}
Jul 21 12:40:20.827408 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:20.827370    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mbfzx\" (UniqueName: \"kubernetes.io/projected/37f02e4b-2a9f-4e44-8c74-d2428bd97ee5-kube-api-access-mbfzx\") pod \"network-check-target-clxmp\" (UID: \"37f02e4b-2a9f-4e44-8c74-d2428bd97ee5\") " pod="openshift-network-diagnostics/network-check-target-clxmp"
Jul 21 12:40:20.827592 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:40:20.827553    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 21 12:40:20.827592 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:40:20.827586    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 21 12:40:20.827698 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:40:20.827599    2556 projected.go:196] Error preparing data for projected volume kube-api-access-mbfzx for pod openshift-network-diagnostics/network-check-target-clxmp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 12:40:20.827698 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:40:20.827664    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/37f02e4b-2a9f-4e44-8c74-d2428bd97ee5-kube-api-access-mbfzx podName:37f02e4b-2a9f-4e44-8c74-d2428bd97ee5 nodeName:}" failed. No retries permitted until 2026-07-21 12:40:52.82764659 +0000 UTC m=+66.131651673 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-mbfzx" (UniqueName: "kubernetes.io/projected/37f02e4b-2a9f-4e44-8c74-d2428bd97ee5-kube-api-access-mbfzx") pod "network-check-target-clxmp" (UID: "37f02e4b-2a9f-4e44-8c74-d2428bd97ee5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 12:40:20.927938 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:20.927901    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d71014d4-ae0b-437e-adca-886a8ff5ac7f-original-pull-secret\") pod \"global-pull-secret-syncer-mchwp\" (UID: \"d71014d4-ae0b-437e-adca-886a8ff5ac7f\") " pod="kube-system/global-pull-secret-syncer-mchwp"
Jul 21 12:40:20.927938 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:20.927940    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/543737a1-6fd3-489b-a2b2-9822825c0007-metrics-certs\") pod \"network-metrics-daemon-jfqn8\" (UID: \"543737a1-6fd3-489b-a2b2-9822825c0007\") " pod="openshift-multus/network-metrics-daemon-jfqn8"
Jul 21 12:40:20.928142 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:40:20.928069    2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 12:40:20.928142 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:40:20.928104    2556 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 21 12:40:20.928142 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:40:20.928128    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/543737a1-6fd3-489b-a2b2-9822825c0007-metrics-certs podName:543737a1-6fd3-489b-a2b2-9822825c0007 nodeName:}" failed. No retries permitted until 2026-07-21 12:40:52.928112794 +0000 UTC m=+66.232117865 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/543737a1-6fd3-489b-a2b2-9822825c0007-metrics-certs") pod "network-metrics-daemon-jfqn8" (UID: "543737a1-6fd3-489b-a2b2-9822825c0007") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 12:40:20.928287 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:40:20.928158    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d71014d4-ae0b-437e-adca-886a8ff5ac7f-original-pull-secret podName:d71014d4-ae0b-437e-adca-886a8ff5ac7f nodeName:}" failed. No retries permitted until 2026-07-21 12:40:52.928145344 +0000 UTC m=+66.232150415 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d71014d4-ae0b-437e-adca-886a8ff5ac7f-original-pull-secret") pod "global-pull-secret-syncer-mchwp" (UID: "d71014d4-ae0b-437e-adca-886a8ff5ac7f") : object "kube-system"/"original-pull-secret" not registered
Jul 21 12:40:21.188815 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:21.188732    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-clxmp"
Jul 21 12:40:21.188978 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:21.188855    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jfqn8"
Jul 21 12:40:21.188978 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:40:21.188865    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-clxmp" podUID="37f02e4b-2a9f-4e44-8c74-d2428bd97ee5"
Jul 21 12:40:21.189115 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:40:21.188985    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jfqn8" podUID="543737a1-6fd3-489b-a2b2-9822825c0007"
Jul 21 12:40:21.189115 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:21.189060    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mchwp"
Jul 21 12:40:21.189220 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:40:21.189128    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mchwp" podUID="d71014d4-ae0b-437e-adca-886a8ff5ac7f"
Jul 21 12:40:21.989875 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:21.989791    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-2.ec2.internal" event="NodeReady"
Jul 21 12:40:21.990390 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:21.989938    2556 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 21 12:40:22.078113 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:22.078080    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-94clr"]
Jul 21 12:40:22.081728 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:22.081703    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-8kzkx"]
Jul 21 12:40:22.081893 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:22.081869    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-94clr"
Jul 21 12:40:22.085798 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:22.085768    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-8kzkx"
Jul 21 12:40:22.086374 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:22.086316    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 21 12:40:22.086502 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:22.086388    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-h5b4t\""
Jul 21 12:40:22.086502 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:22.086477    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 21 12:40:22.094631 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:22.094602    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 21 12:40:22.094631 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:22.094620    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-7n7fg\""
Jul 21 12:40:22.094909 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:22.094894    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 21 12:40:22.094952 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:22.094925    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 21 12:40:22.109558 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:22.109453    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-8kzkx"]
Jul 21 12:40:22.110353 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:22.110334    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-94clr"]
Jul 21 12:40:22.238825 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:22.238793    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/eb9d39d0-f6a2-4050-a227-8aa6061b4799-config-volume\") pod \"dns-default-94clr\" (UID: \"eb9d39d0-f6a2-4050-a227-8aa6061b4799\") " pod="openshift-dns/dns-default-94clr"
Jul 21 12:40:22.238968 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:22.238839    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vh79m\" (UniqueName: \"kubernetes.io/projected/9025d3cc-72d5-4cdb-aad8-79186d9a48ec-kube-api-access-vh79m\") pod \"ingress-canary-8kzkx\" (UID: \"9025d3cc-72d5-4cdb-aad8-79186d9a48ec\") " pod="openshift-ingress-canary/ingress-canary-8kzkx"
Jul 21 12:40:22.238968 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:22.238931    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eb9d39d0-f6a2-4050-a227-8aa6061b4799-tmp-dir\") pod \"dns-default-94clr\" (UID: \"eb9d39d0-f6a2-4050-a227-8aa6061b4799\") " pod="openshift-dns/dns-default-94clr"
Jul 21 12:40:22.239088 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:22.238982    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9025d3cc-72d5-4cdb-aad8-79186d9a48ec-cert\") pod \"ingress-canary-8kzkx\" (UID: \"9025d3cc-72d5-4cdb-aad8-79186d9a48ec\") " pod="openshift-ingress-canary/ingress-canary-8kzkx"
Jul 21 12:40:22.239088 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:22.239022    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/eb9d39d0-f6a2-4050-a227-8aa6061b4799-metrics-tls\") pod \"dns-default-94clr\" (UID: \"eb9d39d0-f6a2-4050-a227-8aa6061b4799\") " pod="openshift-dns/dns-default-94clr"
Jul 21 12:40:22.239088 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:22.239078    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l56s6\" (UniqueName: \"kubernetes.io/projected/eb9d39d0-f6a2-4050-a227-8aa6061b4799-kube-api-access-l56s6\") pod \"dns-default-94clr\" (UID: \"eb9d39d0-f6a2-4050-a227-8aa6061b4799\") " pod="openshift-dns/dns-default-94clr"
Jul 21 12:40:22.340084 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:22.339989    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/eb9d39d0-f6a2-4050-a227-8aa6061b4799-config-volume\") pod \"dns-default-94clr\" (UID: \"eb9d39d0-f6a2-4050-a227-8aa6061b4799\") " pod="openshift-dns/dns-default-94clr"
Jul 21 12:40:22.340084 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:22.340046    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vh79m\" (UniqueName: \"kubernetes.io/projected/9025d3cc-72d5-4cdb-aad8-79186d9a48ec-kube-api-access-vh79m\") pod \"ingress-canary-8kzkx\" (UID: \"9025d3cc-72d5-4cdb-aad8-79186d9a48ec\") " pod="openshift-ingress-canary/ingress-canary-8kzkx"
Jul 21 12:40:22.340084 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:22.340076    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eb9d39d0-f6a2-4050-a227-8aa6061b4799-tmp-dir\") pod \"dns-default-94clr\" (UID: \"eb9d39d0-f6a2-4050-a227-8aa6061b4799\") " pod="openshift-dns/dns-default-94clr"
Jul 21 12:40:22.340319 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:22.340094    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9025d3cc-72d5-4cdb-aad8-79186d9a48ec-cert\") pod \"ingress-canary-8kzkx\" (UID: \"9025d3cc-72d5-4cdb-aad8-79186d9a48ec\") " pod="openshift-ingress-canary/ingress-canary-8kzkx"
Jul 21 12:40:22.340319 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:22.340114    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/eb9d39d0-f6a2-4050-a227-8aa6061b4799-metrics-tls\") pod \"dns-default-94clr\" (UID: \"eb9d39d0-f6a2-4050-a227-8aa6061b4799\") " pod="openshift-dns/dns-default-94clr"
Jul 21 12:40:22.340319 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:22.340129    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l56s6\" (UniqueName: \"kubernetes.io/projected/eb9d39d0-f6a2-4050-a227-8aa6061b4799-kube-api-access-l56s6\") pod \"dns-default-94clr\" (UID: \"eb9d39d0-f6a2-4050-a227-8aa6061b4799\") " pod="openshift-dns/dns-default-94clr"
Jul 21 12:40:22.340524 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:22.340499    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eb9d39d0-f6a2-4050-a227-8aa6061b4799-tmp-dir\") pod \"dns-default-94clr\" (UID: \"eb9d39d0-f6a2-4050-a227-8aa6061b4799\") " pod="openshift-dns/dns-default-94clr"
Jul 21 12:40:22.340624 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:40:22.340608    2556 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 21 12:40:22.340688 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:40:22.340677    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eb9d39d0-f6a2-4050-a227-8aa6061b4799-metrics-tls podName:eb9d39d0-f6a2-4050-a227-8aa6061b4799 nodeName:}" failed. No retries permitted until 2026-07-21 12:40:22.840659545 +0000 UTC m=+36.144664613 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/eb9d39d0-f6a2-4050-a227-8aa6061b4799-metrics-tls") pod "dns-default-94clr" (UID: "eb9d39d0-f6a2-4050-a227-8aa6061b4799") : secret "dns-default-metrics-tls" not found
Jul 21 12:40:22.340771 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:22.340710    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/eb9d39d0-f6a2-4050-a227-8aa6061b4799-config-volume\") pod \"dns-default-94clr\" (UID: \"eb9d39d0-f6a2-4050-a227-8aa6061b4799\") " pod="openshift-dns/dns-default-94clr"
Jul 21 12:40:22.345063 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:22.345039    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9025d3cc-72d5-4cdb-aad8-79186d9a48ec-cert\") pod \"ingress-canary-8kzkx\" (UID: \"9025d3cc-72d5-4cdb-aad8-79186d9a48ec\") " pod="openshift-ingress-canary/ingress-canary-8kzkx"
Jul 21 12:40:22.351152 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:22.351129    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l56s6\" (UniqueName: \"kubernetes.io/projected/eb9d39d0-f6a2-4050-a227-8aa6061b4799-kube-api-access-l56s6\") pod \"dns-default-94clr\" (UID: \"eb9d39d0-f6a2-4050-a227-8aa6061b4799\") " pod="openshift-dns/dns-default-94clr"
Jul 21 12:40:22.351673 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:22.351655    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vh79m\" (UniqueName: \"kubernetes.io/projected/9025d3cc-72d5-4cdb-aad8-79186d9a48ec-kube-api-access-vh79m\") pod \"ingress-canary-8kzkx\" (UID: \"9025d3cc-72d5-4cdb-aad8-79186d9a48ec\") " pod="openshift-ingress-canary/ingress-canary-8kzkx"
Jul 21 12:40:22.401346 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:22.401317    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-8kzkx"
Jul 21 12:40:22.550109 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:22.550035    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-8kzkx"]
Jul 21 12:40:22.555572 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:40:22.555540    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9025d3cc_72d5_4cdb_aad8_79186d9a48ec.slice/crio-60950460e2e720b8d70f6b4047a32b8eb2ad68ecbf2efde73bf23fe282fe9a12 WatchSource:0}: Error finding container 60950460e2e720b8d70f6b4047a32b8eb2ad68ecbf2efde73bf23fe282fe9a12: Status 404 returned error can't find the container with id 60950460e2e720b8d70f6b4047a32b8eb2ad68ecbf2efde73bf23fe282fe9a12
Jul 21 12:40:22.664318 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:22.664240    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-wd79x"]
Jul 21 12:40:22.669394 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:22.669373    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-555d9d7885-rn8m5"]
Jul 21 12:40:22.669550 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:22.669497    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-wd79x"
Jul 21 12:40:22.672865 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:22.672837    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Jul 21 12:40:22.672865 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:22.672857    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-hjdv8\""
Jul 21 12:40:22.672865 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:22.672860    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Jul 21 12:40:22.673224 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:22.673151    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-lrn59"]
Jul 21 12:40:22.673322 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:22.673282    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-555d9d7885-rn8m5"
Jul 21 12:40:22.675420 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:22.675404    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 21 12:40:22.675719 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:22.675702    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 21 12:40:22.675816 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:22.675721    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Jul 21 12:40:22.675877 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:22.675821    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-p2bcw\""
Jul 21 12:40:22.675877 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:22.675702    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Jul 21 12:40:22.677590 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:22.677573    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-lrn59"
Jul 21 12:40:22.678377 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:22.678320    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-wd79x"]
Jul 21 12:40:22.681312 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:22.681259    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-lrn59"]
Jul 21 12:40:22.681712 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:22.681691    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Jul 21 12:40:22.681858 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:22.681834    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Jul 21 12:40:22.681927 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:22.681903    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Jul 21 12:40:22.682105 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:22.682086    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-47fjw\""
Jul 21 12:40:22.684532 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:22.684512    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Jul 21 12:40:22.696155 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:22.696038    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-555d9d7885-rn8m5"]
Jul 21 12:40:22.760398 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:22.760366    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-s7gwh"]
Jul 21 12:40:22.763898 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:22.763873    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-s7gwh"
Jul 21 12:40:22.764457 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:22.764409    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-8zqrp"]
Jul 21 12:40:22.766920 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:22.766889    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Jul 21 12:40:22.766920 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:22.766905    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-rc4tw\""
Jul 21 12:40:22.767100 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:22.767003    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Jul 21 12:40:22.767100 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:22.767004    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Jul 21 12:40:22.767213 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:22.767197    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Jul 21 12:40:22.767644 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:22.767628    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-cldzr"]
Jul 21 12:40:22.767728 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:22.767712    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-8zqrp"
Jul 21 12:40:22.770814 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:22.770797    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-cldzr"
Jul 21 12:40:22.774072 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:22.771939    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Jul 21 12:40:22.774072 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:22.772275    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Jul 21 12:40:22.774072 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:22.772336    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Jul 21 12:40:22.775132 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:22.775115    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-jmk8d\""
Jul 21 12:40:22.775544 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:22.775527    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 21 12:40:22.776226 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:22.776207    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 21 12:40:22.777093 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:22.777066    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Jul 21 12:40:22.777181 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:22.777118    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Jul 21 12:40:22.777233 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:22.777068    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Jul 21 12:40:22.777233 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:22.777226    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-pg5qb\""
Jul 21 12:40:22.777996 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:22.777974    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-s7gwh"]
Jul 21 12:40:22.778788 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:22.778765    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-cldzr"]
Jul 21 12:40:22.787427 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:22.787407    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-8zqrp"]
Jul 21 12:40:22.843116 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:22.843080    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zfvnq\" (UniqueName: \"kubernetes.io/projected/95665b69-15a8-4324-adee-a9bf0a9f0865-kube-api-access-zfvnq\") pod \"volume-data-source-validator-78cf4c6f6-wd79x\" (UID: \"95665b69-15a8-4324-adee-a9bf0a9f0865\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-wd79x"
Jul 21 12:40:22.843116 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:22.843119    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/277500e3-0956-4360-90a2-6512c5725349-serving-cert\") pod \"insights-operator-555d9d7885-rn8m5\" (UID: \"277500e3-0956-4360-90a2-6512c5725349\") " pod="openshift-insights/insights-operator-555d9d7885-rn8m5"
Jul 21 12:40:22.843313 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:22.843198    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/277500e3-0956-4360-90a2-6512c5725349-tmp\") pod \"insights-operator-555d9d7885-rn8m5\" (UID: \"277500e3-0956-4360-90a2-6512c5725349\") " pod="openshift-insights/insights-operator-555d9d7885-rn8m5"
Jul 21 12:40:22.843313 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:22.843244    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mg9x9\" (UniqueName: \"kubernetes.io/projected/277500e3-0956-4360-90a2-6512c5725349-kube-api-access-mg9x9\") pod \"insights-operator-555d9d7885-rn8m5\" (UID: \"277500e3-0956-4360-90a2-6512c5725349\") " pod="openshift-insights/insights-operator-555d9d7885-rn8m5"
Jul 21 12:40:22.843313 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:22.843297    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/277500e3-0956-4360-90a2-6512c5725349-service-ca-bundle\") pod \"insights-operator-555d9d7885-rn8m5\" (UID: \"277500e3-0956-4360-90a2-6512c5725349\") " pod="openshift-insights/insights-operator-555d9d7885-rn8m5"
Jul 21 12:40:22.843421 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:22.843320    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9bf9a989-c13b-4419-84f7-ddab7a99fedf-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-lrn59\" (UID: \"9bf9a989-c13b-4419-84f7-ddab7a99fedf\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-lrn59"
Jul 21 12:40:22.843421 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:22.843363    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kzvqs\" (UniqueName: \"kubernetes.io/projected/9bf9a989-c13b-4419-84f7-ddab7a99fedf-kube-api-access-kzvqs\") pod \"cluster-samples-operator-79b7c55f87-lrn59\" (UID: \"9bf9a989-c13b-4419-84f7-ddab7a99fedf\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-lrn59"
Jul 21 12:40:22.843421 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:22.843386    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/277500e3-0956-4360-90a2-6512c5725349-snapshots\") pod \"insights-operator-555d9d7885-rn8m5\" (UID: \"277500e3-0956-4360-90a2-6512c5725349\") " pod="openshift-insights/insights-operator-555d9d7885-rn8m5"
Jul 21 12:40:22.843421 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:22.843402    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/277500e3-0956-4360-90a2-6512c5725349-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-rn8m5\" (UID: \"277500e3-0956-4360-90a2-6512c5725349\") " pod="openshift-insights/insights-operator-555d9d7885-rn8m5"
Jul 21 12:40:22.843569 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:22.843435    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/eb9d39d0-f6a2-4050-a227-8aa6061b4799-metrics-tls\") pod \"dns-default-94clr\" (UID: \"eb9d39d0-f6a2-4050-a227-8aa6061b4799\") " pod="openshift-dns/dns-default-94clr"
Jul 21 12:40:22.843569 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:40:22.843518    2556 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 21 12:40:22.843660 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:40:22.843573    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eb9d39d0-f6a2-4050-a227-8aa6061b4799-metrics-tls podName:eb9d39d0-f6a2-4050-a227-8aa6061b4799 nodeName:}" failed. No retries permitted until 2026-07-21 12:40:23.843556287 +0000 UTC m=+37.147561357 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/eb9d39d0-f6a2-4050-a227-8aa6061b4799-metrics-tls") pod "dns-default-94clr" (UID: "eb9d39d0-f6a2-4050-a227-8aa6061b4799") : secret "dns-default-metrics-tls" not found
Jul 21 12:40:22.868980 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:22.868953    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-5cffd5944d-w72zg"]
Jul 21 12:40:22.872678 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:22.872649    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5cffd5944d-w72zg"
Jul 21 12:40:22.874891 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:22.874867    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 21 12:40:22.875051 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:22.874951    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-v4mnz\""
Jul 21 12:40:22.875227 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:22.875207    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 21 12:40:22.875354 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:22.875294    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 21 12:40:22.880485 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:22.880459    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 21 12:40:22.884782 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:22.884759    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5cffd5944d-w72zg"]
Jul 21 12:40:22.944293 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:22.944174    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/277500e3-0956-4360-90a2-6512c5725349-service-ca-bundle\") pod \"insights-operator-555d9d7885-rn8m5\" (UID: \"277500e3-0956-4360-90a2-6512c5725349\") " pod="openshift-insights/insights-operator-555d9d7885-rn8m5"
Jul 21 12:40:22.944293 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:22.944222    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8fe9a2da-76ca-4b0c-b86a-4dc3dad01c2e-serving-cert\") pod \"service-ca-operator-5bcbc94c78-s7gwh\" (UID: \"8fe9a2da-76ca-4b0c-b86a-4dc3dad01c2e\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-s7gwh"
Jul 21 12:40:22.944293 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:22.944254    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zjg9r\" (UniqueName: \"kubernetes.io/projected/8fe9a2da-76ca-4b0c-b86a-4dc3dad01c2e-kube-api-access-zjg9r\") pod \"service-ca-operator-5bcbc94c78-s7gwh\" (UID: \"8fe9a2da-76ca-4b0c-b86a-4dc3dad01c2e\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-s7gwh"
Jul 21 12:40:22.944293 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:22.944290    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9bf9a989-c13b-4419-84f7-ddab7a99fedf-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-lrn59\" (UID: \"9bf9a989-c13b-4419-84f7-ddab7a99fedf\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-lrn59"
Jul 21 12:40:22.944612 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:22.944319    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7793bdb2-35aa-476a-a73c-95856526fff3-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-8zqrp\" (UID: \"7793bdb2-35aa-476a-a73c-95856526fff3\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-8zqrp"
Jul 21 12:40:22.944612 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:22.944345    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8fe9a2da-76ca-4b0c-b86a-4dc3dad01c2e-config\") pod \"service-ca-operator-5bcbc94c78-s7gwh\" (UID: \"8fe9a2da-76ca-4b0c-b86a-4dc3dad01c2e\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-s7gwh"
Jul 21 12:40:22.944612 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:40:22.944464    2556 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 21 12:40:22.944612 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:22.944506    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kzvqs\" (UniqueName: \"kubernetes.io/projected/9bf9a989-c13b-4419-84f7-ddab7a99fedf-kube-api-access-kzvqs\") pod \"cluster-samples-operator-79b7c55f87-lrn59\" (UID: \"9bf9a989-c13b-4419-84f7-ddab7a99fedf\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-lrn59"
Jul 21 12:40:22.944612 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:40:22.944542    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9bf9a989-c13b-4419-84f7-ddab7a99fedf-samples-operator-tls podName:9bf9a989-c13b-4419-84f7-ddab7a99fedf nodeName:}" failed. No retries permitted until 2026-07-21 12:40:23.444524978 +0000 UTC m=+36.748530048 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/9bf9a989-c13b-4419-84f7-ddab7a99fedf-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-lrn59" (UID: "9bf9a989-c13b-4419-84f7-ddab7a99fedf") : secret "samples-operator-tls" not found
Jul 21 12:40:22.944612 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:22.944566    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/277500e3-0956-4360-90a2-6512c5725349-snapshots\") pod \"insights-operator-555d9d7885-rn8m5\" (UID: \"277500e3-0956-4360-90a2-6512c5725349\") " pod="openshift-insights/insights-operator-555d9d7885-rn8m5"
Jul 21 12:40:22.944612 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:22.944597    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/277500e3-0956-4360-90a2-6512c5725349-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-rn8m5\" (UID: \"277500e3-0956-4360-90a2-6512c5725349\") " pod="openshift-insights/insights-operator-555d9d7885-rn8m5"
Jul 21 12:40:22.944945 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:22.944641    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rs6jq\" (UniqueName: \"kubernetes.io/projected/7793bdb2-35aa-476a-a73c-95856526fff3-kube-api-access-rs6jq\") pod \"kube-storage-version-migrator-operator-7658d9b749-8zqrp\" (UID: \"7793bdb2-35aa-476a-a73c-95856526fff3\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-8zqrp"
Jul 21 12:40:22.944945 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:22.944669    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a087fecd-1bf9-4449-a9a1-dc1aba2a2ec3-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-cldzr\" (UID: \"a087fecd-1bf9-4449-a9a1-dc1aba2a2ec3\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-cldzr"
Jul 21 12:40:22.944945 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:22.944697    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vbfvm\" (UniqueName: \"kubernetes.io/projected/a087fecd-1bf9-4449-a9a1-dc1aba2a2ec3-kube-api-access-vbfvm\") pod \"cluster-monitoring-operator-76cf8966f8-cldzr\" (UID: \"a087fecd-1bf9-4449-a9a1-dc1aba2a2ec3\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-cldzr"
Jul 21 12:40:22.944945 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:22.944733    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zfvnq\" (UniqueName: \"kubernetes.io/projected/95665b69-15a8-4324-adee-a9bf0a9f0865-kube-api-access-zfvnq\") pod \"volume-data-source-validator-78cf4c6f6-wd79x\" (UID: \"95665b69-15a8-4324-adee-a9bf0a9f0865\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-wd79x"
Jul 21 12:40:22.944945 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:22.944761    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/277500e3-0956-4360-90a2-6512c5725349-serving-cert\") pod \"insights-operator-555d9d7885-rn8m5\" (UID: \"277500e3-0956-4360-90a2-6512c5725349\") " pod="openshift-insights/insights-operator-555d9d7885-rn8m5"
Jul 21 12:40:22.944945 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:22.944841    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/277500e3-0956-4360-90a2-6512c5725349-tmp\") pod \"insights-operator-555d9d7885-rn8m5\" (UID: \"277500e3-0956-4360-90a2-6512c5725349\") " pod="openshift-insights/insights-operator-555d9d7885-rn8m5"
Jul 21 12:40:22.944945 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:22.944875    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mg9x9\" (UniqueName: \"kubernetes.io/projected/277500e3-0956-4360-90a2-6512c5725349-kube-api-access-mg9x9\") pod \"insights-operator-555d9d7885-rn8m5\" (UID: \"277500e3-0956-4360-90a2-6512c5725349\") " pod="openshift-insights/insights-operator-555d9d7885-rn8m5"
Jul 21 12:40:22.944945 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:22.944904    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/277500e3-0956-4360-90a2-6512c5725349-service-ca-bundle\") pod \"insights-operator-555d9d7885-rn8m5\" (UID: \"277500e3-0956-4360-90a2-6512c5725349\") " pod="openshift-insights/insights-operator-555d9d7885-rn8m5"
Jul 21 12:40:22.944945 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:22.944913    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/7793bdb2-35aa-476a-a73c-95856526fff3-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-8zqrp\" (UID: \"7793bdb2-35aa-476a-a73c-95856526fff3\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-8zqrp"
Jul 21 12:40:22.945364 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:22.944979    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/a087fecd-1bf9-4449-a9a1-dc1aba2a2ec3-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-cldzr\" (UID: \"a087fecd-1bf9-4449-a9a1-dc1aba2a2ec3\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-cldzr"
Jul 21 12:40:22.945364 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:22.945215    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/277500e3-0956-4360-90a2-6512c5725349-tmp\") pod \"insights-operator-555d9d7885-rn8m5\" (UID: \"277500e3-0956-4360-90a2-6512c5725349\") " pod="openshift-insights/insights-operator-555d9d7885-rn8m5"
Jul 21 12:40:22.945364 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:22.945259    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/277500e3-0956-4360-90a2-6512c5725349-snapshots\") pod \"insights-operator-555d9d7885-rn8m5\" (UID: \"277500e3-0956-4360-90a2-6512c5725349\") " pod="openshift-insights/insights-operator-555d9d7885-rn8m5"
Jul 21 12:40:22.945494 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:22.945426    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/277500e3-0956-4360-90a2-6512c5725349-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-rn8m5\" (UID: \"277500e3-0956-4360-90a2-6512c5725349\") " pod="openshift-insights/insights-operator-555d9d7885-rn8m5"
Jul 21 12:40:22.948306 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:22.948278    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/277500e3-0956-4360-90a2-6512c5725349-serving-cert\") pod \"insights-operator-555d9d7885-rn8m5\" (UID: \"277500e3-0956-4360-90a2-6512c5725349\") " pod="openshift-insights/insights-operator-555d9d7885-rn8m5"
Jul 21 12:40:22.954471 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:22.954448    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mg9x9\" (UniqueName: \"kubernetes.io/projected/277500e3-0956-4360-90a2-6512c5725349-kube-api-access-mg9x9\") pod \"insights-operator-555d9d7885-rn8m5\" (UID: \"277500e3-0956-4360-90a2-6512c5725349\") " pod="openshift-insights/insights-operator-555d9d7885-rn8m5"
Jul 21 12:40:22.954600 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:22.954562    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kzvqs\" (UniqueName: \"kubernetes.io/projected/9bf9a989-c13b-4419-84f7-ddab7a99fedf-kube-api-access-kzvqs\") pod \"cluster-samples-operator-79b7c55f87-lrn59\" (UID: \"9bf9a989-c13b-4419-84f7-ddab7a99fedf\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-lrn59"
Jul 21 12:40:22.954772 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:22.954749    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zfvnq\" (UniqueName: \"kubernetes.io/projected/95665b69-15a8-4324-adee-a9bf0a9f0865-kube-api-access-zfvnq\") pod \"volume-data-source-validator-78cf4c6f6-wd79x\" (UID: \"95665b69-15a8-4324-adee-a9bf0a9f0865\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-wd79x"
Jul 21 12:40:22.986325 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:22.986293    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-wd79x"
Jul 21 12:40:22.992957 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:22.992927    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-555d9d7885-rn8m5"
Jul 21 12:40:23.045602 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:23.045561    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/352a97a9-8685-40eb-a22e-f0ed802f3cae-image-registry-private-configuration\") pod \"image-registry-5cffd5944d-w72zg\" (UID: \"352a97a9-8685-40eb-a22e-f0ed802f3cae\") " pod="openshift-image-registry/image-registry-5cffd5944d-w72zg"
Jul 21 12:40:23.045773 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:23.045626    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7793bdb2-35aa-476a-a73c-95856526fff3-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-8zqrp\" (UID: \"7793bdb2-35aa-476a-a73c-95856526fff3\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-8zqrp"
Jul 21 12:40:23.045773 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:23.045656    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wqfll\" (UniqueName: \"kubernetes.io/projected/352a97a9-8685-40eb-a22e-f0ed802f3cae-kube-api-access-wqfll\") pod \"image-registry-5cffd5944d-w72zg\" (UID: \"352a97a9-8685-40eb-a22e-f0ed802f3cae\") " pod="openshift-image-registry/image-registry-5cffd5944d-w72zg"
Jul 21 12:40:23.045773 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:23.045683    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a087fecd-1bf9-4449-a9a1-dc1aba2a2ec3-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-cldzr\" (UID: \"a087fecd-1bf9-4449-a9a1-dc1aba2a2ec3\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-cldzr"
Jul 21 12:40:23.045773 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:23.045713    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/352a97a9-8685-40eb-a22e-f0ed802f3cae-registry-certificates\") pod \"image-registry-5cffd5944d-w72zg\" (UID: \"352a97a9-8685-40eb-a22e-f0ed802f3cae\") " pod="openshift-image-registry/image-registry-5cffd5944d-w72zg"
Jul 21 12:40:23.045773 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:23.045738    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/352a97a9-8685-40eb-a22e-f0ed802f3cae-registry-tls\") pod \"image-registry-5cffd5944d-w72zg\" (UID: \"352a97a9-8685-40eb-a22e-f0ed802f3cae\") " pod="openshift-image-registry/image-registry-5cffd5944d-w72zg"
Jul 21 12:40:23.046006 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:23.045793    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/352a97a9-8685-40eb-a22e-f0ed802f3cae-bound-sa-token\") pod \"image-registry-5cffd5944d-w72zg\" (UID: \"352a97a9-8685-40eb-a22e-f0ed802f3cae\") " pod="openshift-image-registry/image-registry-5cffd5944d-w72zg"
Jul 21 12:40:23.046006 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:23.045823    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/7793bdb2-35aa-476a-a73c-95856526fff3-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-8zqrp\" (UID: \"7793bdb2-35aa-476a-a73c-95856526fff3\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-8zqrp"
Jul 21 12:40:23.046006 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:23.045846    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/352a97a9-8685-40eb-a22e-f0ed802f3cae-installation-pull-secrets\") pod \"image-registry-5cffd5944d-w72zg\" (UID: \"352a97a9-8685-40eb-a22e-f0ed802f3cae\") " pod="openshift-image-registry/image-registry-5cffd5944d-w72zg"
Jul 21 12:40:23.046006 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:23.045881    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/a087fecd-1bf9-4449-a9a1-dc1aba2a2ec3-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-cldzr\" (UID: \"a087fecd-1bf9-4449-a9a1-dc1aba2a2ec3\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-cldzr"
Jul 21 12:40:23.046006 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:23.045911    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zjg9r\" (UniqueName: \"kubernetes.io/projected/8fe9a2da-76ca-4b0c-b86a-4dc3dad01c2e-kube-api-access-zjg9r\") pod \"service-ca-operator-5bcbc94c78-s7gwh\" (UID: \"8fe9a2da-76ca-4b0c-b86a-4dc3dad01c2e\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-s7gwh"
Jul 21 12:40:23.046006 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:23.045938    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8fe9a2da-76ca-4b0c-b86a-4dc3dad01c2e-serving-cert\") pod \"service-ca-operator-5bcbc94c78-s7gwh\" (UID: \"8fe9a2da-76ca-4b0c-b86a-4dc3dad01c2e\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-s7gwh"
Jul 21 12:40:23.046006 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:23.045964    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/352a97a9-8685-40eb-a22e-f0ed802f3cae-ca-trust-extracted\") pod \"image-registry-5cffd5944d-w72zg\" (UID: \"352a97a9-8685-40eb-a22e-f0ed802f3cae\") " pod="openshift-image-registry/image-registry-5cffd5944d-w72zg"
Jul 21 12:40:23.046312 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:23.045989    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8fe9a2da-76ca-4b0c-b86a-4dc3dad01c2e-config\") pod \"service-ca-operator-5bcbc94c78-s7gwh\" (UID: \"8fe9a2da-76ca-4b0c-b86a-4dc3dad01c2e\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-s7gwh"
Jul 21 12:40:23.046312 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:23.046187    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/352a97a9-8685-40eb-a22e-f0ed802f3cae-trusted-ca\") pod \"image-registry-5cffd5944d-w72zg\" (UID: \"352a97a9-8685-40eb-a22e-f0ed802f3cae\") " pod="openshift-image-registry/image-registry-5cffd5944d-w72zg"
Jul 21 12:40:23.046543 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:40:23.046491    2556 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 21 12:40:23.046543 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:40:23.046577    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a087fecd-1bf9-4449-a9a1-dc1aba2a2ec3-cluster-monitoring-operator-tls podName:a087fecd-1bf9-4449-a9a1-dc1aba2a2ec3 nodeName:}" failed. No retries permitted until 2026-07-21 12:40:23.546549251 +0000 UTC m=+36.850554320 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/a087fecd-1bf9-4449-a9a1-dc1aba2a2ec3-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-cldzr" (UID: "a087fecd-1bf9-4449-a9a1-dc1aba2a2ec3") : secret "cluster-monitoring-operator-tls" not found
Jul 21 12:40:23.046718 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:23.046666    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/7793bdb2-35aa-476a-a73c-95856526fff3-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-8zqrp\" (UID: \"7793bdb2-35aa-476a-a73c-95856526fff3\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-8zqrp"
Jul 21 12:40:23.046775 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:23.046726    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8fe9a2da-76ca-4b0c-b86a-4dc3dad01c2e-config\") pod \"service-ca-operator-5bcbc94c78-s7gwh\" (UID: \"8fe9a2da-76ca-4b0c-b86a-4dc3dad01c2e\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-s7gwh"
Jul 21 12:40:23.046857 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:23.046831    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rs6jq\" (UniqueName: \"kubernetes.io/projected/7793bdb2-35aa-476a-a73c-95856526fff3-kube-api-access-rs6jq\") pod \"kube-storage-version-migrator-operator-7658d9b749-8zqrp\" (UID: \"7793bdb2-35aa-476a-a73c-95856526fff3\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-8zqrp"
Jul 21 12:40:23.046917 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:23.046877    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vbfvm\" (UniqueName: \"kubernetes.io/projected/a087fecd-1bf9-4449-a9a1-dc1aba2a2ec3-kube-api-access-vbfvm\") pod \"cluster-monitoring-operator-76cf8966f8-cldzr\" (UID: \"a087fecd-1bf9-4449-a9a1-dc1aba2a2ec3\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-cldzr"
Jul 21 12:40:23.046972 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:23.046924    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/a087fecd-1bf9-4449-a9a1-dc1aba2a2ec3-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-cldzr\" (UID: \"a087fecd-1bf9-4449-a9a1-dc1aba2a2ec3\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-cldzr"
Jul 21 12:40:23.048413 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:23.048372    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7793bdb2-35aa-476a-a73c-95856526fff3-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-8zqrp\" (UID: \"7793bdb2-35aa-476a-a73c-95856526fff3\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-8zqrp"
Jul 21 12:40:23.048644 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:23.048625    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8fe9a2da-76ca-4b0c-b86a-4dc3dad01c2e-serving-cert\") pod \"service-ca-operator-5bcbc94c78-s7gwh\" (UID: \"8fe9a2da-76ca-4b0c-b86a-4dc3dad01c2e\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-s7gwh"
Jul 21 12:40:23.058363 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:23.058338    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rs6jq\" (UniqueName: \"kubernetes.io/projected/7793bdb2-35aa-476a-a73c-95856526fff3-kube-api-access-rs6jq\") pod \"kube-storage-version-migrator-operator-7658d9b749-8zqrp\" (UID: \"7793bdb2-35aa-476a-a73c-95856526fff3\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-8zqrp"
Jul 21 12:40:23.058822 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:23.058801    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zjg9r\" (UniqueName: \"kubernetes.io/projected/8fe9a2da-76ca-4b0c-b86a-4dc3dad01c2e-kube-api-access-zjg9r\") pod \"service-ca-operator-5bcbc94c78-s7gwh\" (UID: \"8fe9a2da-76ca-4b0c-b86a-4dc3dad01c2e\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-s7gwh"
Jul 21 12:40:23.063997 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:23.063976    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vbfvm\" (UniqueName: \"kubernetes.io/projected/a087fecd-1bf9-4449-a9a1-dc1aba2a2ec3-kube-api-access-vbfvm\") pod \"cluster-monitoring-operator-76cf8966f8-cldzr\" (UID: \"a087fecd-1bf9-4449-a9a1-dc1aba2a2ec3\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-cldzr"
Jul 21 12:40:23.079700 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:23.079677    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-s7gwh"
Jul 21 12:40:23.089123 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:23.089096    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-8zqrp"
Jul 21 12:40:23.147686 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:23.147648    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/352a97a9-8685-40eb-a22e-f0ed802f3cae-image-registry-private-configuration\") pod \"image-registry-5cffd5944d-w72zg\" (UID: \"352a97a9-8685-40eb-a22e-f0ed802f3cae\") " pod="openshift-image-registry/image-registry-5cffd5944d-w72zg"
Jul 21 12:40:23.147862 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:23.147710    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wqfll\" (UniqueName: \"kubernetes.io/projected/352a97a9-8685-40eb-a22e-f0ed802f3cae-kube-api-access-wqfll\") pod \"image-registry-5cffd5944d-w72zg\" (UID: \"352a97a9-8685-40eb-a22e-f0ed802f3cae\") " pod="openshift-image-registry/image-registry-5cffd5944d-w72zg"
Jul 21 12:40:23.147862 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:23.147750    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/352a97a9-8685-40eb-a22e-f0ed802f3cae-registry-certificates\") pod \"image-registry-5cffd5944d-w72zg\" (UID: \"352a97a9-8685-40eb-a22e-f0ed802f3cae\") " pod="openshift-image-registry/image-registry-5cffd5944d-w72zg"
Jul 21 12:40:23.147862 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:23.147773    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/352a97a9-8685-40eb-a22e-f0ed802f3cae-registry-tls\") pod \"image-registry-5cffd5944d-w72zg\" (UID: \"352a97a9-8685-40eb-a22e-f0ed802f3cae\") " pod="openshift-image-registry/image-registry-5cffd5944d-w72zg"
Jul 21 12:40:23.148040 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:23.147852    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/352a97a9-8685-40eb-a22e-f0ed802f3cae-bound-sa-token\") pod \"image-registry-5cffd5944d-w72zg\" (UID: \"352a97a9-8685-40eb-a22e-f0ed802f3cae\") " pod="openshift-image-registry/image-registry-5cffd5944d-w72zg"
Jul 21 12:40:23.148040 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:40:23.147895    2556 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 21 12:40:23.148040 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:23.147901    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/352a97a9-8685-40eb-a22e-f0ed802f3cae-installation-pull-secrets\") pod \"image-registry-5cffd5944d-w72zg\" (UID: \"352a97a9-8685-40eb-a22e-f0ed802f3cae\") " pod="openshift-image-registry/image-registry-5cffd5944d-w72zg"
Jul 21 12:40:23.148040 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:40:23.147910    2556 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5cffd5944d-w72zg: secret "image-registry-tls" not found
Jul 21 12:40:23.148040 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:23.147937    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/352a97a9-8685-40eb-a22e-f0ed802f3cae-ca-trust-extracted\") pod \"image-registry-5cffd5944d-w72zg\" (UID: \"352a97a9-8685-40eb-a22e-f0ed802f3cae\") " pod="openshift-image-registry/image-registry-5cffd5944d-w72zg"
Jul 21 12:40:23.148040 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:40:23.147970    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/352a97a9-8685-40eb-a22e-f0ed802f3cae-registry-tls podName:352a97a9-8685-40eb-a22e-f0ed802f3cae nodeName:}" failed. No retries permitted until 2026-07-21 12:40:23.647950891 +0000 UTC m=+36.951955976 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/352a97a9-8685-40eb-a22e-f0ed802f3cae-registry-tls") pod "image-registry-5cffd5944d-w72zg" (UID: "352a97a9-8685-40eb-a22e-f0ed802f3cae") : secret "image-registry-tls" not found
Jul 21 12:40:23.148040 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:23.147991    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/352a97a9-8685-40eb-a22e-f0ed802f3cae-trusted-ca\") pod \"image-registry-5cffd5944d-w72zg\" (UID: \"352a97a9-8685-40eb-a22e-f0ed802f3cae\") " pod="openshift-image-registry/image-registry-5cffd5944d-w72zg"
Jul 21 12:40:23.148968 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:23.148941    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/352a97a9-8685-40eb-a22e-f0ed802f3cae-ca-trust-extracted\") pod \"image-registry-5cffd5944d-w72zg\" (UID: \"352a97a9-8685-40eb-a22e-f0ed802f3cae\") " pod="openshift-image-registry/image-registry-5cffd5944d-w72zg"
Jul 21 12:40:23.149099 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:23.149069    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/352a97a9-8685-40eb-a22e-f0ed802f3cae-registry-certificates\") pod \"image-registry-5cffd5944d-w72zg\" (UID: \"352a97a9-8685-40eb-a22e-f0ed802f3cae\") " pod="openshift-image-registry/image-registry-5cffd5944d-w72zg"
Jul 21 12:40:23.149540 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:23.149517    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/352a97a9-8685-40eb-a22e-f0ed802f3cae-trusted-ca\") pod \"image-registry-5cffd5944d-w72zg\" (UID: \"352a97a9-8685-40eb-a22e-f0ed802f3cae\") " pod="openshift-image-registry/image-registry-5cffd5944d-w72zg"
Jul 21 12:40:23.150675 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:23.150658    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/352a97a9-8685-40eb-a22e-f0ed802f3cae-installation-pull-secrets\") pod \"image-registry-5cffd5944d-w72zg\" (UID: \"352a97a9-8685-40eb-a22e-f0ed802f3cae\") " pod="openshift-image-registry/image-registry-5cffd5944d-w72zg"
Jul 21 12:40:23.150768 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:23.150750    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/352a97a9-8685-40eb-a22e-f0ed802f3cae-image-registry-private-configuration\") pod \"image-registry-5cffd5944d-w72zg\" (UID: \"352a97a9-8685-40eb-a22e-f0ed802f3cae\") " pod="openshift-image-registry/image-registry-5cffd5944d-w72zg"
Jul 21 12:40:23.160475 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:23.160440    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/352a97a9-8685-40eb-a22e-f0ed802f3cae-bound-sa-token\") pod \"image-registry-5cffd5944d-w72zg\" (UID: \"352a97a9-8685-40eb-a22e-f0ed802f3cae\") " pod="openshift-image-registry/image-registry-5cffd5944d-w72zg"
Jul 21 12:40:23.161219 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:23.161197    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wqfll\" (UniqueName: \"kubernetes.io/projected/352a97a9-8685-40eb-a22e-f0ed802f3cae-kube-api-access-wqfll\") pod \"image-registry-5cffd5944d-w72zg\" (UID: \"352a97a9-8685-40eb-a22e-f0ed802f3cae\") " pod="openshift-image-registry/image-registry-5cffd5944d-w72zg"
Jul 21 12:40:23.188255 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:23.188225    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-clxmp"
Jul 21 12:40:23.188255 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:23.188257    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mchwp"
Jul 21 12:40:23.188437 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:23.188226    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jfqn8"
Jul 21 12:40:23.193809 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:23.193784    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 21 12:40:23.193809 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:23.193807    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 21 12:40:23.194164 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:23.194145    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-tqbcg\""
Jul 21 12:40:23.194272 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:23.194214    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 21 12:40:23.195100 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:23.194864    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 21 12:40:23.195100 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:23.194896    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-rhfvk\""
Jul 21 12:40:23.347529 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:23.347486    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-8kzkx" event={"ID":"9025d3cc-72d5-4cdb-aad8-79186d9a48ec","Type":"ContainerStarted","Data":"60950460e2e720b8d70f6b4047a32b8eb2ad68ecbf2efde73bf23fe282fe9a12"}
Jul 21 12:40:23.451159 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:23.451053    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9bf9a989-c13b-4419-84f7-ddab7a99fedf-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-lrn59\" (UID: \"9bf9a989-c13b-4419-84f7-ddab7a99fedf\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-lrn59"
Jul 21 12:40:23.451308 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:40:23.451223    2556 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 21 12:40:23.451308 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:40:23.451297    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9bf9a989-c13b-4419-84f7-ddab7a99fedf-samples-operator-tls podName:9bf9a989-c13b-4419-84f7-ddab7a99fedf nodeName:}" failed. No retries permitted until 2026-07-21 12:40:24.451280497 +0000 UTC m=+37.755285585 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/9bf9a989-c13b-4419-84f7-ddab7a99fedf-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-lrn59" (UID: "9bf9a989-c13b-4419-84f7-ddab7a99fedf") : secret "samples-operator-tls" not found
Jul 21 12:40:23.551987 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:23.551946    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a087fecd-1bf9-4449-a9a1-dc1aba2a2ec3-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-cldzr\" (UID: \"a087fecd-1bf9-4449-a9a1-dc1aba2a2ec3\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-cldzr"
Jul 21 12:40:23.552171 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:40:23.552101    2556 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 21 12:40:23.552237 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:40:23.552183    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a087fecd-1bf9-4449-a9a1-dc1aba2a2ec3-cluster-monitoring-operator-tls podName:a087fecd-1bf9-4449-a9a1-dc1aba2a2ec3 nodeName:}" failed. No retries permitted until 2026-07-21 12:40:24.552168093 +0000 UTC m=+37.856173161 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/a087fecd-1bf9-4449-a9a1-dc1aba2a2ec3-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-cldzr" (UID: "a087fecd-1bf9-4449-a9a1-dc1aba2a2ec3") : secret "cluster-monitoring-operator-tls" not found
Jul 21 12:40:23.653317 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:23.653278    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/352a97a9-8685-40eb-a22e-f0ed802f3cae-registry-tls\") pod \"image-registry-5cffd5944d-w72zg\" (UID: \"352a97a9-8685-40eb-a22e-f0ed802f3cae\") " pod="openshift-image-registry/image-registry-5cffd5944d-w72zg"
Jul 21 12:40:23.653490 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:40:23.653463    2556 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 21 12:40:23.653490 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:40:23.653490    2556 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5cffd5944d-w72zg: secret "image-registry-tls" not found
Jul 21 12:40:23.653669 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:40:23.653560    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/352a97a9-8685-40eb-a22e-f0ed802f3cae-registry-tls podName:352a97a9-8685-40eb-a22e-f0ed802f3cae nodeName:}" failed. No retries permitted until 2026-07-21 12:40:24.653543559 +0000 UTC m=+37.957548631 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/352a97a9-8685-40eb-a22e-f0ed802f3cae-registry-tls") pod "image-registry-5cffd5944d-w72zg" (UID: "352a97a9-8685-40eb-a22e-f0ed802f3cae") : secret "image-registry-tls" not found
Jul 21 12:40:23.855372 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:23.855331    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/eb9d39d0-f6a2-4050-a227-8aa6061b4799-metrics-tls\") pod \"dns-default-94clr\" (UID: \"eb9d39d0-f6a2-4050-a227-8aa6061b4799\") " pod="openshift-dns/dns-default-94clr"
Jul 21 12:40:23.855558 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:40:23.855467    2556 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 21 12:40:23.855558 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:40:23.855535    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eb9d39d0-f6a2-4050-a227-8aa6061b4799-metrics-tls podName:eb9d39d0-f6a2-4050-a227-8aa6061b4799 nodeName:}" failed. No retries permitted until 2026-07-21 12:40:25.85551683 +0000 UTC m=+39.159521898 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/eb9d39d0-f6a2-4050-a227-8aa6061b4799-metrics-tls") pod "dns-default-94clr" (UID: "eb9d39d0-f6a2-4050-a227-8aa6061b4799") : secret "dns-default-metrics-tls" not found
Jul 21 12:40:24.461243 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:24.460616    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9bf9a989-c13b-4419-84f7-ddab7a99fedf-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-lrn59\" (UID: \"9bf9a989-c13b-4419-84f7-ddab7a99fedf\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-lrn59"
Jul 21 12:40:24.461243 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:40:24.460812    2556 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 21 12:40:24.461243 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:40:24.460877    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9bf9a989-c13b-4419-84f7-ddab7a99fedf-samples-operator-tls podName:9bf9a989-c13b-4419-84f7-ddab7a99fedf nodeName:}" failed. No retries permitted until 2026-07-21 12:40:26.460858762 +0000 UTC m=+39.764863833 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/9bf9a989-c13b-4419-84f7-ddab7a99fedf-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-lrn59" (UID: "9bf9a989-c13b-4419-84f7-ddab7a99fedf") : secret "samples-operator-tls" not found
Jul 21 12:40:24.478194 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:24.478105    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-8zqrp"]
Jul 21 12:40:24.482612 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:40:24.482580    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7793bdb2_35aa_476a_a73c_95856526fff3.slice/crio-3e58e118ebf4c0ba67770d11b5ad556479bfb170059942d8129b8ff4333823c3 WatchSource:0}: Error finding container 3e58e118ebf4c0ba67770d11b5ad556479bfb170059942d8129b8ff4333823c3: Status 404 returned error can't find the container with id 3e58e118ebf4c0ba67770d11b5ad556479bfb170059942d8129b8ff4333823c3
Jul 21 12:40:24.482814 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:24.482782    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-555d9d7885-rn8m5"]
Jul 21 12:40:24.495839 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:24.495775    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-wd79x"]
Jul 21 12:40:24.497608 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:24.497586    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-s7gwh"]
Jul 21 12:40:24.512199 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:40:24.512173    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8fe9a2da_76ca_4b0c_b86a_4dc3dad01c2e.slice/crio-292920574b432791226519d1c4ac96cb630e43a2c7ad29a9e55cb357dc39e120 WatchSource:0}: Error finding container 292920574b432791226519d1c4ac96cb630e43a2c7ad29a9e55cb357dc39e120: Status 404 returned error can't find the container with id 292920574b432791226519d1c4ac96cb630e43a2c7ad29a9e55cb357dc39e120
Jul 21 12:40:24.561447 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:24.561419    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a087fecd-1bf9-4449-a9a1-dc1aba2a2ec3-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-cldzr\" (UID: \"a087fecd-1bf9-4449-a9a1-dc1aba2a2ec3\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-cldzr"
Jul 21 12:40:24.561625 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:40:24.561603    2556 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 21 12:40:24.561698 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:40:24.561687    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a087fecd-1bf9-4449-a9a1-dc1aba2a2ec3-cluster-monitoring-operator-tls podName:a087fecd-1bf9-4449-a9a1-dc1aba2a2ec3 nodeName:}" failed. No retries permitted until 2026-07-21 12:40:26.561665011 +0000 UTC m=+39.865670083 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/a087fecd-1bf9-4449-a9a1-dc1aba2a2ec3-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-cldzr" (UID: "a087fecd-1bf9-4449-a9a1-dc1aba2a2ec3") : secret "cluster-monitoring-operator-tls" not found
Jul 21 12:40:24.662701 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:24.662671    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/352a97a9-8685-40eb-a22e-f0ed802f3cae-registry-tls\") pod \"image-registry-5cffd5944d-w72zg\" (UID: \"352a97a9-8685-40eb-a22e-f0ed802f3cae\") " pod="openshift-image-registry/image-registry-5cffd5944d-w72zg"
Jul 21 12:40:24.662883 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:40:24.662855    2556 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 21 12:40:24.662883 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:40:24.662883    2556 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5cffd5944d-w72zg: secret "image-registry-tls" not found
Jul 21 12:40:24.663084 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:40:24.662938    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/352a97a9-8685-40eb-a22e-f0ed802f3cae-registry-tls podName:352a97a9-8685-40eb-a22e-f0ed802f3cae nodeName:}" failed. No retries permitted until 2026-07-21 12:40:26.662923137 +0000 UTC m=+39.966928204 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/352a97a9-8685-40eb-a22e-f0ed802f3cae-registry-tls") pod "image-registry-5cffd5944d-w72zg" (UID: "352a97a9-8685-40eb-a22e-f0ed802f3cae") : secret "image-registry-tls" not found
Jul 21 12:40:25.357297 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:25.357233    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-s7gwh" event={"ID":"8fe9a2da-76ca-4b0c-b86a-4dc3dad01c2e","Type":"ContainerStarted","Data":"292920574b432791226519d1c4ac96cb630e43a2c7ad29a9e55cb357dc39e120"}
Jul 21 12:40:25.360330 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:25.360273    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-wd79x" event={"ID":"95665b69-15a8-4324-adee-a9bf0a9f0865","Type":"ContainerStarted","Data":"c42da06a271236e1630d0df91698e0b0a336f8b957265f7f4f84cbbe7f9c0ea1"}
Jul 21 12:40:25.361771 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:25.361739    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-rn8m5" event={"ID":"277500e3-0956-4360-90a2-6512c5725349","Type":"ContainerStarted","Data":"c3b38932971f797793e84d561294a4cc8836de27b05eb33f33de62fb08aa0078"}
Jul 21 12:40:25.369123 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:25.369091    2556 generic.go:356] "Generic (PLEG): container finished" podID="c2aa47a3-b6ce-4807-8e97-aaf79dc1d941" containerID="5d42e2651cd7d3fad84867e57dc2e220531a96eab4a224c8991b02c5c396fb61" exitCode=0
Jul 21 12:40:25.369229 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:25.369179    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vbvlg" event={"ID":"c2aa47a3-b6ce-4807-8e97-aaf79dc1d941","Type":"ContainerDied","Data":"5d42e2651cd7d3fad84867e57dc2e220531a96eab4a224c8991b02c5c396fb61"}
Jul 21 12:40:25.371482 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:25.371457    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-8zqrp" event={"ID":"7793bdb2-35aa-476a-a73c-95856526fff3","Type":"ContainerStarted","Data":"3e58e118ebf4c0ba67770d11b5ad556479bfb170059942d8129b8ff4333823c3"}
Jul 21 12:40:25.873549 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:25.873312    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/eb9d39d0-f6a2-4050-a227-8aa6061b4799-metrics-tls\") pod \"dns-default-94clr\" (UID: \"eb9d39d0-f6a2-4050-a227-8aa6061b4799\") " pod="openshift-dns/dns-default-94clr"
Jul 21 12:40:25.873980 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:40:25.873807    2556 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 21 12:40:25.873980 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:40:25.873881    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eb9d39d0-f6a2-4050-a227-8aa6061b4799-metrics-tls podName:eb9d39d0-f6a2-4050-a227-8aa6061b4799 nodeName:}" failed. No retries permitted until 2026-07-21 12:40:29.873860323 +0000 UTC m=+43.177865391 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/eb9d39d0-f6a2-4050-a227-8aa6061b4799-metrics-tls") pod "dns-default-94clr" (UID: "eb9d39d0-f6a2-4050-a227-8aa6061b4799") : secret "dns-default-metrics-tls" not found
Jul 21 12:40:26.385048 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:26.384992    2556 generic.go:356] "Generic (PLEG): container finished" podID="c2aa47a3-b6ce-4807-8e97-aaf79dc1d941" containerID="d211e7eb6bf10dc9775487b2f90f79ff864f1d65f078fb952f1476f22bc7eee1" exitCode=0
Jul 21 12:40:26.385272 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:26.385255    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vbvlg" event={"ID":"c2aa47a3-b6ce-4807-8e97-aaf79dc1d941","Type":"ContainerDied","Data":"d211e7eb6bf10dc9775487b2f90f79ff864f1d65f078fb952f1476f22bc7eee1"}
Jul 21 12:40:26.483985 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:26.483272    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9bf9a989-c13b-4419-84f7-ddab7a99fedf-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-lrn59\" (UID: \"9bf9a989-c13b-4419-84f7-ddab7a99fedf\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-lrn59"
Jul 21 12:40:26.483985 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:40:26.483406    2556 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 21 12:40:26.483985 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:40:26.483465    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9bf9a989-c13b-4419-84f7-ddab7a99fedf-samples-operator-tls podName:9bf9a989-c13b-4419-84f7-ddab7a99fedf nodeName:}" failed. No retries permitted until 2026-07-21 12:40:30.483449317 +0000 UTC m=+43.787454385 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/9bf9a989-c13b-4419-84f7-ddab7a99fedf-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-lrn59" (UID: "9bf9a989-c13b-4419-84f7-ddab7a99fedf") : secret "samples-operator-tls" not found
Jul 21 12:40:26.584123 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:26.584087    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a087fecd-1bf9-4449-a9a1-dc1aba2a2ec3-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-cldzr\" (UID: \"a087fecd-1bf9-4449-a9a1-dc1aba2a2ec3\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-cldzr"
Jul 21 12:40:26.584266 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:40:26.584197    2556 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 21 12:40:26.584266 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:40:26.584260    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a087fecd-1bf9-4449-a9a1-dc1aba2a2ec3-cluster-monitoring-operator-tls podName:a087fecd-1bf9-4449-a9a1-dc1aba2a2ec3 nodeName:}" failed. No retries permitted until 2026-07-21 12:40:30.584244856 +0000 UTC m=+43.888249924 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/a087fecd-1bf9-4449-a9a1-dc1aba2a2ec3-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-cldzr" (UID: "a087fecd-1bf9-4449-a9a1-dc1aba2a2ec3") : secret "cluster-monitoring-operator-tls" not found
Jul 21 12:40:26.684542 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:26.684451    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/352a97a9-8685-40eb-a22e-f0ed802f3cae-registry-tls\") pod \"image-registry-5cffd5944d-w72zg\" (UID: \"352a97a9-8685-40eb-a22e-f0ed802f3cae\") " pod="openshift-image-registry/image-registry-5cffd5944d-w72zg"
Jul 21 12:40:26.684682 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:40:26.684572    2556 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 21 12:40:26.684682 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:40:26.684592    2556 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5cffd5944d-w72zg: secret "image-registry-tls" not found
Jul 21 12:40:26.684682 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:40:26.684661    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/352a97a9-8685-40eb-a22e-f0ed802f3cae-registry-tls podName:352a97a9-8685-40eb-a22e-f0ed802f3cae nodeName:}" failed. No retries permitted until 2026-07-21 12:40:30.68464137 +0000 UTC m=+43.988646438 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/352a97a9-8685-40eb-a22e-f0ed802f3cae-registry-tls") pod "image-registry-5cffd5944d-w72zg" (UID: "352a97a9-8685-40eb-a22e-f0ed802f3cae") : secret "image-registry-tls" not found
Jul 21 12:40:29.908457 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:29.908429    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/eb9d39d0-f6a2-4050-a227-8aa6061b4799-metrics-tls\") pod \"dns-default-94clr\" (UID: \"eb9d39d0-f6a2-4050-a227-8aa6061b4799\") " pod="openshift-dns/dns-default-94clr"
Jul 21 12:40:29.908872 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:40:29.908582    2556 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 21 12:40:29.908872 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:40:29.908642    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eb9d39d0-f6a2-4050-a227-8aa6061b4799-metrics-tls podName:eb9d39d0-f6a2-4050-a227-8aa6061b4799 nodeName:}" failed. No retries permitted until 2026-07-21 12:40:37.908623315 +0000 UTC m=+51.212628386 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/eb9d39d0-f6a2-4050-a227-8aa6061b4799-metrics-tls") pod "dns-default-94clr" (UID: "eb9d39d0-f6a2-4050-a227-8aa6061b4799") : secret "dns-default-metrics-tls" not found
Jul 21 12:40:30.396705 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:30.396490    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-8zqrp" event={"ID":"7793bdb2-35aa-476a-a73c-95856526fff3","Type":"ContainerStarted","Data":"60ef621461807bf3b9765b9750a1a9cbdea5304128d66d43eba30ab000cce689"}
Jul 21 12:40:30.398071 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:30.398045    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-8kzkx" event={"ID":"9025d3cc-72d5-4cdb-aad8-79186d9a48ec","Type":"ContainerStarted","Data":"0a7298c5a437196fe6a8cab259e1720898968f777e0ff889cb2aad463abafbe6"}
Jul 21 12:40:30.399625 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:30.399594    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-s7gwh" event={"ID":"8fe9a2da-76ca-4b0c-b86a-4dc3dad01c2e","Type":"ContainerStarted","Data":"7f419aca93b3d3b96be5179a341b4ffce4f3a0597d3e6df39468af1ef79b3382"}
Jul 21 12:40:30.401428 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:30.401406    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-wd79x" event={"ID":"95665b69-15a8-4324-adee-a9bf0a9f0865","Type":"ContainerStarted","Data":"ea8afca70a0d127cddc90980542ebe6ce769e08790aa5951f0adc4914b0c35ca"}
Jul 21 12:40:30.407355 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:30.407332    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-rn8m5" event={"ID":"277500e3-0956-4360-90a2-6512c5725349","Type":"ContainerStarted","Data":"646cf7365c3c41f4debcf3fab58a7612379d8d61823a46e59e255a01fae54d7c"}
Jul 21 12:40:30.411008 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:30.410987    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vbvlg" event={"ID":"c2aa47a3-b6ce-4807-8e97-aaf79dc1d941","Type":"ContainerStarted","Data":"45616becb288528e8c160d2fe3b17991fcb2a2ab8af1dc081506c0873edb8c45"}
Jul 21 12:40:30.414428 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:30.414390    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-8zqrp" podStartSLOduration=3.183698584 podStartE2EDuration="8.414378015s" totalImagesPullingTime="5.230679431s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:40:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:40:30.413810532 +0000 UTC m=+43.717815624" watchObservedRunningTime="2026-07-21 12:40:30.414378015 +0000 UTC m=+43.718383104"
Jul 21 12:40:30.435776 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:30.435735    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-vbvlg" podStartSLOduration=12.655274777 podStartE2EDuration="43.43572345s" totalImagesPullingTime="30.780448673s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 12:39:47 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:40:30.43544381 +0000 UTC m=+43.739448900" watchObservedRunningTime="2026-07-21 12:40:30.43572345 +0000 UTC m=+43.739728542"
Jul 21 12:40:30.451533 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:30.451490    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-8kzkx" podStartSLOduration=1.2917171330000001 podStartE2EDuration="8.451475241s" totalImagesPullingTime="7.159758108s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:40:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:40:30.451139292 +0000 UTC m=+43.755144384" watchObservedRunningTime="2026-07-21 12:40:30.451475241 +0000 UTC m=+43.755480329"
Jul 21 12:40:30.487495 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:30.486246    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-s7gwh" podStartSLOduration=3.2857643420000002 podStartE2EDuration="8.486229831s" totalImagesPullingTime="5.200465489s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:40:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:40:30.485995285 +0000 UTC m=+43.790000375" watchObservedRunningTime="2026-07-21 12:40:30.486229831 +0000 UTC m=+43.790234917"
Jul 21 12:40:30.487495 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:30.487292    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-wd79x" podStartSLOduration=3.286309886 podStartE2EDuration="8.48728228s" totalImagesPullingTime="5.200972394s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:40:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:40:30.468081651 +0000 UTC m=+43.772086773" watchObservedRunningTime="2026-07-21 12:40:30.48728228 +0000 UTC m=+43.791287351"
Jul 21 12:40:30.507701 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:30.507661    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-555d9d7885-rn8m5" podStartSLOduration=3.286452738 podStartE2EDuration="8.507645326s" totalImagesPullingTime="5.221192588s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:40:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:40:30.50720654 +0000 UTC m=+43.811211650" watchObservedRunningTime="2026-07-21 12:40:30.507645326 +0000 UTC m=+43.811650420"
Jul 21 12:40:30.514114 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:30.513607    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9bf9a989-c13b-4419-84f7-ddab7a99fedf-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-lrn59\" (UID: \"9bf9a989-c13b-4419-84f7-ddab7a99fedf\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-lrn59"
Jul 21 12:40:30.514114 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:40:30.513719    2556 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 21 12:40:30.514114 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:40:30.513769    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9bf9a989-c13b-4419-84f7-ddab7a99fedf-samples-operator-tls podName:9bf9a989-c13b-4419-84f7-ddab7a99fedf nodeName:}" failed. No retries permitted until 2026-07-21 12:40:38.513755599 +0000 UTC m=+51.817760666 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/9bf9a989-c13b-4419-84f7-ddab7a99fedf-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-lrn59" (UID: "9bf9a989-c13b-4419-84f7-ddab7a99fedf") : secret "samples-operator-tls" not found
Jul 21 12:40:30.614210 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:30.614181    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a087fecd-1bf9-4449-a9a1-dc1aba2a2ec3-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-cldzr\" (UID: \"a087fecd-1bf9-4449-a9a1-dc1aba2a2ec3\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-cldzr"
Jul 21 12:40:30.614351 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:40:30.614331    2556 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 21 12:40:30.614414 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:40:30.614404    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a087fecd-1bf9-4449-a9a1-dc1aba2a2ec3-cluster-monitoring-operator-tls podName:a087fecd-1bf9-4449-a9a1-dc1aba2a2ec3 nodeName:}" failed. No retries permitted until 2026-07-21 12:40:38.614385355 +0000 UTC m=+51.918390427 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/a087fecd-1bf9-4449-a9a1-dc1aba2a2ec3-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-cldzr" (UID: "a087fecd-1bf9-4449-a9a1-dc1aba2a2ec3") : secret "cluster-monitoring-operator-tls" not found
Jul 21 12:40:30.715618 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:30.715518    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/352a97a9-8685-40eb-a22e-f0ed802f3cae-registry-tls\") pod \"image-registry-5cffd5944d-w72zg\" (UID: \"352a97a9-8685-40eb-a22e-f0ed802f3cae\") " pod="openshift-image-registry/image-registry-5cffd5944d-w72zg"
Jul 21 12:40:30.715778 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:40:30.715741    2556 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 21 12:40:30.715778 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:40:30.715759    2556 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5cffd5944d-w72zg: secret "image-registry-tls" not found
Jul 21 12:40:30.715891 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:40:30.715821    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/352a97a9-8685-40eb-a22e-f0ed802f3cae-registry-tls podName:352a97a9-8685-40eb-a22e-f0ed802f3cae nodeName:}" failed. No retries permitted until 2026-07-21 12:40:38.715801567 +0000 UTC m=+52.019806655 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/352a97a9-8685-40eb-a22e-f0ed802f3cae-registry-tls") pod "image-registry-5cffd5944d-w72zg" (UID: "352a97a9-8685-40eb-a22e-f0ed802f3cae") : secret "image-registry-tls" not found
Jul 21 12:40:30.749729 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:30.749701    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-9j6f5"]
Jul 21 12:40:30.767034 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:30.766987    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-9j6f5"]
Jul 21 12:40:30.767195 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:30.767123    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-9j6f5"
Jul 21 12:40:30.769217 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:30.769195    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-6mtb2\""
Jul 21 12:40:30.769217 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:30.769207    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 21 12:40:30.769421 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:30.769351    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 21 12:40:30.816170 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:30.816142    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/dfe654b2-569e-4c88-a935-0f82db6839e5-crio-socket\") pod \"insights-runtime-extractor-9j6f5\" (UID: \"dfe654b2-569e-4c88-a935-0f82db6839e5\") " pod="openshift-insights/insights-runtime-extractor-9j6f5"
Jul 21 12:40:30.816293 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:30.816176    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/dfe654b2-569e-4c88-a935-0f82db6839e5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9j6f5\" (UID: \"dfe654b2-569e-4c88-a935-0f82db6839e5\") " pod="openshift-insights/insights-runtime-extractor-9j6f5"
Jul 21 12:40:30.816293 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:30.816203    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rb6gs\" (UniqueName: \"kubernetes.io/projected/dfe654b2-569e-4c88-a935-0f82db6839e5-kube-api-access-rb6gs\") pod \"insights-runtime-extractor-9j6f5\" (UID: \"dfe654b2-569e-4c88-a935-0f82db6839e5\") " pod="openshift-insights/insights-runtime-extractor-9j6f5"
Jul 21 12:40:30.816383 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:30.816291    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/dfe654b2-569e-4c88-a935-0f82db6839e5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9j6f5\" (UID: \"dfe654b2-569e-4c88-a935-0f82db6839e5\") " pod="openshift-insights/insights-runtime-extractor-9j6f5"
Jul 21 12:40:30.816427 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:30.816412    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/dfe654b2-569e-4c88-a935-0f82db6839e5-data-volume\") pod \"insights-runtime-extractor-9j6f5\" (UID: \"dfe654b2-569e-4c88-a935-0f82db6839e5\") " pod="openshift-insights/insights-runtime-extractor-9j6f5"
Jul 21 12:40:30.917560 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:30.917527    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/dfe654b2-569e-4c88-a935-0f82db6839e5-data-volume\") pod \"insights-runtime-extractor-9j6f5\" (UID: \"dfe654b2-569e-4c88-a935-0f82db6839e5\") " pod="openshift-insights/insights-runtime-extractor-9j6f5"
Jul 21 12:40:30.918049 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:30.917607    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/dfe654b2-569e-4c88-a935-0f82db6839e5-crio-socket\") pod \"insights-runtime-extractor-9j6f5\" (UID: \"dfe654b2-569e-4c88-a935-0f82db6839e5\") " pod="openshift-insights/insights-runtime-extractor-9j6f5"
Jul 21 12:40:30.918049 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:30.917644    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/dfe654b2-569e-4c88-a935-0f82db6839e5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9j6f5\" (UID: \"dfe654b2-569e-4c88-a935-0f82db6839e5\") " pod="openshift-insights/insights-runtime-extractor-9j6f5"
Jul 21 12:40:30.918049 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:30.917670    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rb6gs\" (UniqueName: \"kubernetes.io/projected/dfe654b2-569e-4c88-a935-0f82db6839e5-kube-api-access-rb6gs\") pod \"insights-runtime-extractor-9j6f5\" (UID: \"dfe654b2-569e-4c88-a935-0f82db6839e5\") " pod="openshift-insights/insights-runtime-extractor-9j6f5"
Jul 21 12:40:30.918049 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:30.917718    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/dfe654b2-569e-4c88-a935-0f82db6839e5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9j6f5\" (UID: \"dfe654b2-569e-4c88-a935-0f82db6839e5\") " pod="openshift-insights/insights-runtime-extractor-9j6f5"
Jul 21 12:40:30.918049 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:30.917846    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/dfe654b2-569e-4c88-a935-0f82db6839e5-crio-socket\") pod \"insights-runtime-extractor-9j6f5\" (UID: \"dfe654b2-569e-4c88-a935-0f82db6839e5\") " pod="openshift-insights/insights-runtime-extractor-9j6f5"
Jul 21 12:40:30.918049 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:40:30.917853    2556 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 21 12:40:30.918049 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:30.917869    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/dfe654b2-569e-4c88-a935-0f82db6839e5-data-volume\") pod \"insights-runtime-extractor-9j6f5\" (UID: \"dfe654b2-569e-4c88-a935-0f82db6839e5\") " pod="openshift-insights/insights-runtime-extractor-9j6f5"
Jul 21 12:40:30.918049 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:40:30.917930    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/dfe654b2-569e-4c88-a935-0f82db6839e5-kube-rbac-proxy-cm podName:dfe654b2-569e-4c88-a935-0f82db6839e5 nodeName:}" failed. No retries permitted until 2026-07-21 12:40:31.417911751 +0000 UTC m=+44.721916836 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/dfe654b2-569e-4c88-a935-0f82db6839e5-kube-rbac-proxy-cm") pod "insights-runtime-extractor-9j6f5" (UID: "dfe654b2-569e-4c88-a935-0f82db6839e5") : configmap "kube-rbac-proxy" not found
Jul 21 12:40:30.918049 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:40:30.917966    2556 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 21 12:40:30.918049 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:40:30.918042    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dfe654b2-569e-4c88-a935-0f82db6839e5-insights-runtime-extractor-tls podName:dfe654b2-569e-4c88-a935-0f82db6839e5 nodeName:}" failed. No retries permitted until 2026-07-21 12:40:31.417999048 +0000 UTC m=+44.722004123 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/dfe654b2-569e-4c88-a935-0f82db6839e5-insights-runtime-extractor-tls") pod "insights-runtime-extractor-9j6f5" (UID: "dfe654b2-569e-4c88-a935-0f82db6839e5") : secret "insights-runtime-extractor-tls" not found
Jul 21 12:40:30.930129 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:30.930096    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rb6gs\" (UniqueName: \"kubernetes.io/projected/dfe654b2-569e-4c88-a935-0f82db6839e5-kube-api-access-rb6gs\") pod \"insights-runtime-extractor-9j6f5\" (UID: \"dfe654b2-569e-4c88-a935-0f82db6839e5\") " pod="openshift-insights/insights-runtime-extractor-9j6f5"
Jul 21 12:40:31.420824 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:31.420795    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/dfe654b2-569e-4c88-a935-0f82db6839e5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9j6f5\" (UID: \"dfe654b2-569e-4c88-a935-0f82db6839e5\") " pod="openshift-insights/insights-runtime-extractor-9j6f5"
Jul 21 12:40:31.421001 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:31.420844    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/dfe654b2-569e-4c88-a935-0f82db6839e5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9j6f5\" (UID: \"dfe654b2-569e-4c88-a935-0f82db6839e5\") " pod="openshift-insights/insights-runtime-extractor-9j6f5"
Jul 21 12:40:31.421001 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:40:31.420919    2556 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 21 12:40:31.421001 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:40:31.420930    2556 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 21 12:40:31.421001 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:40:31.420987    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dfe654b2-569e-4c88-a935-0f82db6839e5-insights-runtime-extractor-tls podName:dfe654b2-569e-4c88-a935-0f82db6839e5 nodeName:}" failed. No retries permitted until 2026-07-21 12:40:32.420973102 +0000 UTC m=+45.724978171 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/dfe654b2-569e-4c88-a935-0f82db6839e5-insights-runtime-extractor-tls") pod "insights-runtime-extractor-9j6f5" (UID: "dfe654b2-569e-4c88-a935-0f82db6839e5") : secret "insights-runtime-extractor-tls" not found
Jul 21 12:40:31.421001 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:40:31.421001    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/dfe654b2-569e-4c88-a935-0f82db6839e5-kube-rbac-proxy-cm podName:dfe654b2-569e-4c88-a935-0f82db6839e5 nodeName:}" failed. No retries permitted until 2026-07-21 12:40:32.420994908 +0000 UTC m=+45.724999976 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/dfe654b2-569e-4c88-a935-0f82db6839e5-kube-rbac-proxy-cm") pod "insights-runtime-extractor-9j6f5" (UID: "dfe654b2-569e-4c88-a935-0f82db6839e5") : configmap "kube-rbac-proxy" not found
Jul 21 12:40:32.429709 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:32.429677    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/dfe654b2-569e-4c88-a935-0f82db6839e5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9j6f5\" (UID: \"dfe654b2-569e-4c88-a935-0f82db6839e5\") " pod="openshift-insights/insights-runtime-extractor-9j6f5"
Jul 21 12:40:32.430140 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:32.429788    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/dfe654b2-569e-4c88-a935-0f82db6839e5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9j6f5\" (UID: \"dfe654b2-569e-4c88-a935-0f82db6839e5\") " pod="openshift-insights/insights-runtime-extractor-9j6f5"
Jul 21 12:40:32.430140 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:40:32.429801    2556 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 21 12:40:32.430140 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:40:32.429879    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/dfe654b2-569e-4c88-a935-0f82db6839e5-kube-rbac-proxy-cm podName:dfe654b2-569e-4c88-a935-0f82db6839e5 nodeName:}" failed. No retries permitted until 2026-07-21 12:40:34.429854896 +0000 UTC m=+47.733859965 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/dfe654b2-569e-4c88-a935-0f82db6839e5-kube-rbac-proxy-cm") pod "insights-runtime-extractor-9j6f5" (UID: "dfe654b2-569e-4c88-a935-0f82db6839e5") : configmap "kube-rbac-proxy" not found
Jul 21 12:40:32.430140 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:40:32.429880    2556 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 21 12:40:32.430140 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:40:32.429931    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dfe654b2-569e-4c88-a935-0f82db6839e5-insights-runtime-extractor-tls podName:dfe654b2-569e-4c88-a935-0f82db6839e5 nodeName:}" failed. No retries permitted until 2026-07-21 12:40:34.429920021 +0000 UTC m=+47.733925089 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/dfe654b2-569e-4c88-a935-0f82db6839e5-insights-runtime-extractor-tls") pod "insights-runtime-extractor-9j6f5" (UID: "dfe654b2-569e-4c88-a935-0f82db6839e5") : secret "insights-runtime-extractor-tls" not found
Jul 21 12:40:32.934032 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:32.933988    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-z24v8_e5faf5b4-5f7e-44dd-9503-3cc15f07533e/dns-node-resolver/0.log"
Jul 21 12:40:33.535086 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:33.535063    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-76wr9_5adf7abd-fea9-4a2a-8f96-a73bd0a3404a/node-ca/0.log"
Jul 21 12:40:34.334678 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:34.334648    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-8kzkx_9025d3cc-72d5-4cdb-aad8-79186d9a48ec/serve-healthcheck-canary/0.log"
Jul 21 12:40:34.442992 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:34.442964    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/dfe654b2-569e-4c88-a935-0f82db6839e5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9j6f5\" (UID: \"dfe654b2-569e-4c88-a935-0f82db6839e5\") " pod="openshift-insights/insights-runtime-extractor-9j6f5"
Jul 21 12:40:34.443133 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:34.443006    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/dfe654b2-569e-4c88-a935-0f82db6839e5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9j6f5\" (UID: \"dfe654b2-569e-4c88-a935-0f82db6839e5\") " pod="openshift-insights/insights-runtime-extractor-9j6f5"
Jul 21 12:40:34.443133 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:40:34.443121    2556 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 21 12:40:34.443207 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:40:34.443123    2556 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 21 12:40:34.443207 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:40:34.443173    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/dfe654b2-569e-4c88-a935-0f82db6839e5-kube-rbac-proxy-cm podName:dfe654b2-569e-4c88-a935-0f82db6839e5 nodeName:}" failed. No retries permitted until 2026-07-21 12:40:38.443156407 +0000 UTC m=+51.747161502 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/dfe654b2-569e-4c88-a935-0f82db6839e5-kube-rbac-proxy-cm") pod "insights-runtime-extractor-9j6f5" (UID: "dfe654b2-569e-4c88-a935-0f82db6839e5") : configmap "kube-rbac-proxy" not found
Jul 21 12:40:34.443281 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:40:34.443215    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dfe654b2-569e-4c88-a935-0f82db6839e5-insights-runtime-extractor-tls podName:dfe654b2-569e-4c88-a935-0f82db6839e5 nodeName:}" failed. No retries permitted until 2026-07-21 12:40:38.443203195 +0000 UTC m=+51.747208267 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/dfe654b2-569e-4c88-a935-0f82db6839e5-insights-runtime-extractor-tls") pod "insights-runtime-extractor-9j6f5" (UID: "dfe654b2-569e-4c88-a935-0f82db6839e5") : secret "insights-runtime-extractor-tls" not found
Jul 21 12:40:37.969176 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:37.969133    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/eb9d39d0-f6a2-4050-a227-8aa6061b4799-metrics-tls\") pod \"dns-default-94clr\" (UID: \"eb9d39d0-f6a2-4050-a227-8aa6061b4799\") " pod="openshift-dns/dns-default-94clr"
Jul 21 12:40:37.969645 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:40:37.969291    2556 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 21 12:40:37.969645 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:40:37.969362    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eb9d39d0-f6a2-4050-a227-8aa6061b4799-metrics-tls podName:eb9d39d0-f6a2-4050-a227-8aa6061b4799 nodeName:}" failed. No retries permitted until 2026-07-21 12:40:53.969345409 +0000 UTC m=+67.273350478 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/eb9d39d0-f6a2-4050-a227-8aa6061b4799-metrics-tls") pod "dns-default-94clr" (UID: "eb9d39d0-f6a2-4050-a227-8aa6061b4799") : secret "dns-default-metrics-tls" not found
Jul 21 12:40:38.472826 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:38.472787    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/dfe654b2-569e-4c88-a935-0f82db6839e5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9j6f5\" (UID: \"dfe654b2-569e-4c88-a935-0f82db6839e5\") " pod="openshift-insights/insights-runtime-extractor-9j6f5"
Jul 21 12:40:38.472826 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:38.472829    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/dfe654b2-569e-4c88-a935-0f82db6839e5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9j6f5\" (UID: \"dfe654b2-569e-4c88-a935-0f82db6839e5\") " pod="openshift-insights/insights-runtime-extractor-9j6f5"
Jul 21 12:40:38.473051 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:40:38.472898    2556 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 21 12:40:38.473051 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:40:38.472923    2556 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 21 12:40:38.473051 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:40:38.472965    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/dfe654b2-569e-4c88-a935-0f82db6839e5-kube-rbac-proxy-cm podName:dfe654b2-569e-4c88-a935-0f82db6839e5 nodeName:}" failed. No retries permitted until 2026-07-21 12:40:46.472951937 +0000 UTC m=+59.776957005 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/dfe654b2-569e-4c88-a935-0f82db6839e5-kube-rbac-proxy-cm") pod "insights-runtime-extractor-9j6f5" (UID: "dfe654b2-569e-4c88-a935-0f82db6839e5") : configmap "kube-rbac-proxy" not found
Jul 21 12:40:38.473051 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:40:38.473043    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dfe654b2-569e-4c88-a935-0f82db6839e5-insights-runtime-extractor-tls podName:dfe654b2-569e-4c88-a935-0f82db6839e5 nodeName:}" failed. No retries permitted until 2026-07-21 12:40:46.473032385 +0000 UTC m=+59.777037452 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/dfe654b2-569e-4c88-a935-0f82db6839e5-insights-runtime-extractor-tls") pod "insights-runtime-extractor-9j6f5" (UID: "dfe654b2-569e-4c88-a935-0f82db6839e5") : secret "insights-runtime-extractor-tls" not found
Jul 21 12:40:38.573631 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:38.573602    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9bf9a989-c13b-4419-84f7-ddab7a99fedf-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-lrn59\" (UID: \"9bf9a989-c13b-4419-84f7-ddab7a99fedf\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-lrn59"
Jul 21 12:40:38.573750 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:40:38.573735    2556 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 21 12:40:38.573801 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:40:38.573789    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9bf9a989-c13b-4419-84f7-ddab7a99fedf-samples-operator-tls podName:9bf9a989-c13b-4419-84f7-ddab7a99fedf nodeName:}" failed. No retries permitted until 2026-07-21 12:40:54.573774645 +0000 UTC m=+67.877779712 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/9bf9a989-c13b-4419-84f7-ddab7a99fedf-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-lrn59" (UID: "9bf9a989-c13b-4419-84f7-ddab7a99fedf") : secret "samples-operator-tls" not found
Jul 21 12:40:38.674691 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:38.674654    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a087fecd-1bf9-4449-a9a1-dc1aba2a2ec3-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-cldzr\" (UID: \"a087fecd-1bf9-4449-a9a1-dc1aba2a2ec3\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-cldzr"
Jul 21 12:40:38.674814 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:40:38.674773    2556 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 21 12:40:38.674881 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:40:38.674834    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a087fecd-1bf9-4449-a9a1-dc1aba2a2ec3-cluster-monitoring-operator-tls podName:a087fecd-1bf9-4449-a9a1-dc1aba2a2ec3 nodeName:}" failed. No retries permitted until 2026-07-21 12:40:54.674818493 +0000 UTC m=+67.978823574 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/a087fecd-1bf9-4449-a9a1-dc1aba2a2ec3-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-cldzr" (UID: "a087fecd-1bf9-4449-a9a1-dc1aba2a2ec3") : secret "cluster-monitoring-operator-tls" not found
Jul 21 12:40:38.775539 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:38.775509    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/352a97a9-8685-40eb-a22e-f0ed802f3cae-registry-tls\") pod \"image-registry-5cffd5944d-w72zg\" (UID: \"352a97a9-8685-40eb-a22e-f0ed802f3cae\") " pod="openshift-image-registry/image-registry-5cffd5944d-w72zg"
Jul 21 12:40:38.775655 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:40:38.775599    2556 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 21 12:40:38.775655 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:40:38.775609    2556 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5cffd5944d-w72zg: secret "image-registry-tls" not found
Jul 21 12:40:38.775655 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:40:38.775649    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/352a97a9-8685-40eb-a22e-f0ed802f3cae-registry-tls podName:352a97a9-8685-40eb-a22e-f0ed802f3cae nodeName:}" failed. No retries permitted until 2026-07-21 12:40:54.775638277 +0000 UTC m=+68.079643346 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/352a97a9-8685-40eb-a22e-f0ed802f3cae-registry-tls") pod "image-registry-5cffd5944d-w72zg" (UID: "352a97a9-8685-40eb-a22e-f0ed802f3cae") : secret "image-registry-tls" not found
Jul 21 12:40:46.540421 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:46.540379    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/dfe654b2-569e-4c88-a935-0f82db6839e5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9j6f5\" (UID: \"dfe654b2-569e-4c88-a935-0f82db6839e5\") " pod="openshift-insights/insights-runtime-extractor-9j6f5"
Jul 21 12:40:46.540421 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:46.540426    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/dfe654b2-569e-4c88-a935-0f82db6839e5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9j6f5\" (UID: \"dfe654b2-569e-4c88-a935-0f82db6839e5\") " pod="openshift-insights/insights-runtime-extractor-9j6f5"
Jul 21 12:40:46.540991 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:40:46.540559    2556 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 21 12:40:46.540991 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:40:46.540646    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dfe654b2-569e-4c88-a935-0f82db6839e5-insights-runtime-extractor-tls podName:dfe654b2-569e-4c88-a935-0f82db6839e5 nodeName:}" failed. No retries permitted until 2026-07-21 12:41:02.540626059 +0000 UTC m=+75.844631130 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/dfe654b2-569e-4c88-a935-0f82db6839e5-insights-runtime-extractor-tls") pod "insights-runtime-extractor-9j6f5" (UID: "dfe654b2-569e-4c88-a935-0f82db6839e5") : secret "insights-runtime-extractor-tls" not found
Jul 21 12:40:46.540991 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:46.540887    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/dfe654b2-569e-4c88-a935-0f82db6839e5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9j6f5\" (UID: \"dfe654b2-569e-4c88-a935-0f82db6839e5\") " pod="openshift-insights/insights-runtime-extractor-9j6f5"
Jul 21 12:40:47.336687 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:47.336655    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-488zb"
Jul 21 12:40:52.888749 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:52.888707    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mbfzx\" (UniqueName: \"kubernetes.io/projected/37f02e4b-2a9f-4e44-8c74-d2428bd97ee5-kube-api-access-mbfzx\") pod \"network-check-target-clxmp\" (UID: \"37f02e4b-2a9f-4e44-8c74-d2428bd97ee5\") " pod="openshift-network-diagnostics/network-check-target-clxmp"
Jul 21 12:40:52.891753 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:52.891733    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 21 12:40:52.902323 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:52.902303    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 21 12:40:52.913270 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:52.913250    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mbfzx\" (UniqueName: \"kubernetes.io/projected/37f02e4b-2a9f-4e44-8c74-d2428bd97ee5-kube-api-access-mbfzx\") pod \"network-check-target-clxmp\" (UID: \"37f02e4b-2a9f-4e44-8c74-d2428bd97ee5\") " pod="openshift-network-diagnostics/network-check-target-clxmp"
Jul 21 12:40:52.989605 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:52.989581    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d71014d4-ae0b-437e-adca-886a8ff5ac7f-original-pull-secret\") pod \"global-pull-secret-syncer-mchwp\" (UID: \"d71014d4-ae0b-437e-adca-886a8ff5ac7f\") " pod="kube-system/global-pull-secret-syncer-mchwp"
Jul 21 12:40:52.989696 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:52.989613    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/543737a1-6fd3-489b-a2b2-9822825c0007-metrics-certs\") pod \"network-metrics-daemon-jfqn8\" (UID: \"543737a1-6fd3-489b-a2b2-9822825c0007\") " pod="openshift-multus/network-metrics-daemon-jfqn8"
Jul 21 12:40:52.992910 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:52.992892    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 21 12:40:52.993353 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:52.993333    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 21 12:40:53.002903 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:53.002877    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/543737a1-6fd3-489b-a2b2-9822825c0007-metrics-certs\") pod \"network-metrics-daemon-jfqn8\" (UID: \"543737a1-6fd3-489b-a2b2-9822825c0007\") " pod="openshift-multus/network-metrics-daemon-jfqn8"
Jul 21 12:40:53.003227 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:53.003199    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d71014d4-ae0b-437e-adca-886a8ff5ac7f-original-pull-secret\") pod \"global-pull-secret-syncer-mchwp\" (UID: \"d71014d4-ae0b-437e-adca-886a8ff5ac7f\") " pod="kube-system/global-pull-secret-syncer-mchwp"
Jul 21 12:40:53.203909 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:53.203851    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-rhfvk\""
Jul 21 12:40:53.208076 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:53.208051    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mchwp"
Jul 21 12:40:53.211750 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:53.211735    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-clxmp"
Jul 21 12:40:53.216916 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:53.216898    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-tqbcg\""
Jul 21 12:40:53.225049 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:53.225023    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jfqn8"
Jul 21 12:40:53.361102 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:53.361071    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-clxmp"]
Jul 21 12:40:53.365275 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:40:53.365246    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37f02e4b_2a9f_4e44_8c74_d2428bd97ee5.slice/crio-89ddc850cef7c729bde106fcf78f200eee841419ca8a9f81aee03a85c04fc256 WatchSource:0}: Error finding container 89ddc850cef7c729bde106fcf78f200eee841419ca8a9f81aee03a85c04fc256: Status 404 returned error can't find the container with id 89ddc850cef7c729bde106fcf78f200eee841419ca8a9f81aee03a85c04fc256
Jul 21 12:40:53.376379 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:53.376332    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-mchwp"]
Jul 21 12:40:53.378616 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:40:53.378596    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd71014d4_ae0b_437e_adca_886a8ff5ac7f.slice/crio-e26c6d9b5e4bba6a2b170c2a200d07a61cf5840b8bda7eab1bb61231f3f3340c WatchSource:0}: Error finding container e26c6d9b5e4bba6a2b170c2a200d07a61cf5840b8bda7eab1bb61231f3f3340c: Status 404 returned error can't find the container with id e26c6d9b5e4bba6a2b170c2a200d07a61cf5840b8bda7eab1bb61231f3f3340c
Jul 21 12:40:53.396789 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:53.396768    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-jfqn8"]
Jul 21 12:40:53.399079 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:40:53.399060    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod543737a1_6fd3_489b_a2b2_9822825c0007.slice/crio-2019a2cbb2d9cc32e0405cb0b371ff5010906356fecf660546bd73ed0aad9f42 WatchSource:0}: Error finding container 2019a2cbb2d9cc32e0405cb0b371ff5010906356fecf660546bd73ed0aad9f42: Status 404 returned error can't find the container with id 2019a2cbb2d9cc32e0405cb0b371ff5010906356fecf660546bd73ed0aad9f42
Jul 21 12:40:53.469563 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:53.469508    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-clxmp" event={"ID":"37f02e4b-2a9f-4e44-8c74-d2428bd97ee5","Type":"ContainerStarted","Data":"89ddc850cef7c729bde106fcf78f200eee841419ca8a9f81aee03a85c04fc256"}
Jul 21 12:40:53.470592 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:53.470563    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-mchwp" event={"ID":"d71014d4-ae0b-437e-adca-886a8ff5ac7f","Type":"ContainerStarted","Data":"e26c6d9b5e4bba6a2b170c2a200d07a61cf5840b8bda7eab1bb61231f3f3340c"}
Jul 21 12:40:53.471591 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:53.471569    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-jfqn8" event={"ID":"543737a1-6fd3-489b-a2b2-9822825c0007","Type":"ContainerStarted","Data":"2019a2cbb2d9cc32e0405cb0b371ff5010906356fecf660546bd73ed0aad9f42"}
Jul 21 12:40:53.997469 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:53.997441    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/eb9d39d0-f6a2-4050-a227-8aa6061b4799-metrics-tls\") pod \"dns-default-94clr\" (UID: \"eb9d39d0-f6a2-4050-a227-8aa6061b4799\") " pod="openshift-dns/dns-default-94clr"
Jul 21 12:40:54.000460 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:54.000428    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/eb9d39d0-f6a2-4050-a227-8aa6061b4799-metrics-tls\") pod \"dns-default-94clr\" (UID: \"eb9d39d0-f6a2-4050-a227-8aa6061b4799\") " pod="openshift-dns/dns-default-94clr"
Jul 21 12:40:54.198985 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:54.198444    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-h5b4t\""
Jul 21 12:40:54.207175 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:54.206899    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-94clr"
Jul 21 12:40:54.370757 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:54.370717    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-94clr"]
Jul 21 12:40:54.602810 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:54.602667    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9bf9a989-c13b-4419-84f7-ddab7a99fedf-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-lrn59\" (UID: \"9bf9a989-c13b-4419-84f7-ddab7a99fedf\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-lrn59"
Jul 21 12:40:54.605833 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:54.605804    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9bf9a989-c13b-4419-84f7-ddab7a99fedf-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-lrn59\" (UID: \"9bf9a989-c13b-4419-84f7-ddab7a99fedf\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-lrn59"
Jul 21 12:40:54.703528 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:54.703498    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a087fecd-1bf9-4449-a9a1-dc1aba2a2ec3-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-cldzr\" (UID: \"a087fecd-1bf9-4449-a9a1-dc1aba2a2ec3\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-cldzr"
Jul 21 12:40:54.706267 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:54.706244    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a087fecd-1bf9-4449-a9a1-dc1aba2a2ec3-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-cldzr\" (UID: \"a087fecd-1bf9-4449-a9a1-dc1aba2a2ec3\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-cldzr"
Jul 21 12:40:54.802073 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:54.802048    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-47fjw\""
Jul 21 12:40:54.804519 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:54.804084    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/352a97a9-8685-40eb-a22e-f0ed802f3cae-registry-tls\") pod \"image-registry-5cffd5944d-w72zg\" (UID: \"352a97a9-8685-40eb-a22e-f0ed802f3cae\") " pod="openshift-image-registry/image-registry-5cffd5944d-w72zg"
Jul 21 12:40:54.808459 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:54.808421    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/352a97a9-8685-40eb-a22e-f0ed802f3cae-registry-tls\") pod \"image-registry-5cffd5944d-w72zg\" (UID: \"352a97a9-8685-40eb-a22e-f0ed802f3cae\") " pod="openshift-image-registry/image-registry-5cffd5944d-w72zg"
Jul 21 12:40:54.809233 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:54.809214    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-lrn59"
Jul 21 12:40:54.902619 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:54.902388    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-jmk8d\""
Jul 21 12:40:54.906289 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:54.906096    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-cldzr"
Jul 21 12:40:54.987753 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:54.987728    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-v4mnz\""
Jul 21 12:40:54.995401 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:54.995375    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5cffd5944d-w72zg"
Jul 21 12:40:55.006498 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:55.005980    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-lrn59"]
Jul 21 12:40:55.108939 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:55.108876    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-cldzr"]
Jul 21 12:40:55.113146 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:40:55.113119    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda087fecd_1bf9_4449_a9a1_dc1aba2a2ec3.slice/crio-057a8fdb18909e144dbc153c20cd98eab453d79f66fd8d2a0fd4a51922003844 WatchSource:0}: Error finding container 057a8fdb18909e144dbc153c20cd98eab453d79f66fd8d2a0fd4a51922003844: Status 404 returned error can't find the container with id 057a8fdb18909e144dbc153c20cd98eab453d79f66fd8d2a0fd4a51922003844
Jul 21 12:40:55.198648 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:55.198629    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5cffd5944d-w72zg"]
Jul 21 12:40:55.484910 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:55.484113    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5cffd5944d-w72zg" event={"ID":"352a97a9-8685-40eb-a22e-f0ed802f3cae","Type":"ContainerStarted","Data":"e89dcf0c8419a2c4e90db56c76b9a48ad05b04c64bb2a4b93ae6282077d8a44d"}
Jul 21 12:40:55.484910 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:55.484177    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5cffd5944d-w72zg" event={"ID":"352a97a9-8685-40eb-a22e-f0ed802f3cae","Type":"ContainerStarted","Data":"dbb2111af8944e3aec67351e8d99a4bdcf64e9a17ab3bd7fc1d2ea4c8ba01735"}
Jul 21 12:40:55.484910 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:55.484753    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-5cffd5944d-w72zg"
Jul 21 12:40:55.489408 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:55.489370    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-jfqn8" event={"ID":"543737a1-6fd3-489b-a2b2-9822825c0007","Type":"ContainerStarted","Data":"7b15b1b6e25afa82def628e55df6cca58fd368c8abc3e530db06c6a85a91d1f4"}
Jul 21 12:40:55.489539 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:55.489432    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-jfqn8" event={"ID":"543737a1-6fd3-489b-a2b2-9822825c0007","Type":"ContainerStarted","Data":"8d5ada58e7ec17ec1b44105880b0d384eea04ebf4354db36571c177d244e23d0"}
Jul 21 12:40:55.491231 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:55.491205    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-cldzr" event={"ID":"a087fecd-1bf9-4449-a9a1-dc1aba2a2ec3","Type":"ContainerStarted","Data":"057a8fdb18909e144dbc153c20cd98eab453d79f66fd8d2a0fd4a51922003844"}
Jul 21 12:40:55.492882 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:55.492859    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-lrn59" event={"ID":"9bf9a989-c13b-4419-84f7-ddab7a99fedf","Type":"ContainerStarted","Data":"a5f93786c0ef413e66e11e8f6d34b5f840f00ee595dba81bff297941327e13c2"}
Jul 21 12:40:55.494373 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:55.494314    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-94clr" event={"ID":"eb9d39d0-f6a2-4050-a227-8aa6061b4799","Type":"ContainerStarted","Data":"098316c519db231e5135a891513491abcd1b95963855615b8eb774d5a55bfa6d"}
Jul 21 12:40:55.510399 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:55.510308    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-5cffd5944d-w72zg" podStartSLOduration=33.510290591 podStartE2EDuration="33.510290591s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:40:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:40:55.509097189 +0000 UTC m=+68.813102303" watchObservedRunningTime="2026-07-21 12:40:55.510290591 +0000 UTC m=+68.814295684"
Jul 21 12:40:55.528582 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:55.528535    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-jfqn8" podStartSLOduration=66.273591433 podStartE2EDuration="1m7.528517066s" totalImagesPullingTime="1.254925633s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:39:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:40:55.528274828 +0000 UTC m=+68.832279941" watchObservedRunningTime="2026-07-21 12:40:55.528517066 +0000 UTC m=+68.832522158"
Jul 21 12:40:59.397138 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:59.397110    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-765f54f9d6-t2xgw"]
Jul 21 12:40:59.421546 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:59.421521    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-765f54f9d6-t2xgw"]
Jul 21 12:40:59.421705 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:59.421630    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-765f54f9d6-t2xgw"
Jul 21 12:40:59.424744 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:59.424719    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-ggv7n\""
Jul 21 12:40:59.424867 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:59.424719    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 21 12:40:59.425536 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:59.425517    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 21 12:40:59.425629 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:59.425570    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Jul 21 12:40:59.430699 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:59.430680    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 21 12:40:59.442633 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:59.442614    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5cffd5944d-w72zg"]
Jul 21 12:40:59.544438 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:59.544398    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/38f8b753-80af-418e-b400-ebfe518a3a71-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-765f54f9d6-t2xgw\" (UID: \"38f8b753-80af-418e-b400-ebfe518a3a71\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-765f54f9d6-t2xgw"
Jul 21 12:40:59.544619 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:59.544512    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9fg4p\" (UniqueName: \"kubernetes.io/projected/38f8b753-80af-418e-b400-ebfe518a3a71-kube-api-access-9fg4p\") pod \"managed-serviceaccount-addon-agent-765f54f9d6-t2xgw\" (UID: \"38f8b753-80af-418e-b400-ebfe518a3a71\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-765f54f9d6-t2xgw"
Jul 21 12:40:59.576913 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:59.576890    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-f4c5cdd74-l4dbz"]
Jul 21 12:40:59.590280 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:59.590255    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-f4c5cdd74-l4dbz"
Jul 21 12:40:59.595082 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:59.595061    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-f4c5cdd74-l4dbz"]
Jul 21 12:40:59.645259 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:59.645234    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/38f8b753-80af-418e-b400-ebfe518a3a71-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-765f54f9d6-t2xgw\" (UID: \"38f8b753-80af-418e-b400-ebfe518a3a71\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-765f54f9d6-t2xgw"
Jul 21 12:40:59.645380 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:59.645276    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9fg4p\" (UniqueName: \"kubernetes.io/projected/38f8b753-80af-418e-b400-ebfe518a3a71-kube-api-access-9fg4p\") pod \"managed-serviceaccount-addon-agent-765f54f9d6-t2xgw\" (UID: \"38f8b753-80af-418e-b400-ebfe518a3a71\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-765f54f9d6-t2xgw"
Jul 21 12:40:59.647885 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:59.647839    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/38f8b753-80af-418e-b400-ebfe518a3a71-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-765f54f9d6-t2xgw\" (UID: \"38f8b753-80af-418e-b400-ebfe518a3a71\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-765f54f9d6-t2xgw"
Jul 21 12:40:59.659108 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:59.659084    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9fg4p\" (UniqueName: \"kubernetes.io/projected/38f8b753-80af-418e-b400-ebfe518a3a71-kube-api-access-9fg4p\") pod \"managed-serviceaccount-addon-agent-765f54f9d6-t2xgw\" (UID: \"38f8b753-80af-418e-b400-ebfe518a3a71\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-765f54f9d6-t2xgw"
Jul 21 12:40:59.743107 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:59.743084    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-765f54f9d6-t2xgw"
Jul 21 12:40:59.746224 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:59.746203    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/755f9627-4c80-4597-ac00-670d261bb1ba-ca-trust-extracted\") pod \"image-registry-f4c5cdd74-l4dbz\" (UID: \"755f9627-4c80-4597-ac00-670d261bb1ba\") " pod="openshift-image-registry/image-registry-f4c5cdd74-l4dbz"
Jul 21 12:40:59.746330 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:59.746239    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/755f9627-4c80-4597-ac00-670d261bb1ba-registry-certificates\") pod \"image-registry-f4c5cdd74-l4dbz\" (UID: \"755f9627-4c80-4597-ac00-670d261bb1ba\") " pod="openshift-image-registry/image-registry-f4c5cdd74-l4dbz"
Jul 21 12:40:59.746330 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:59.746274    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mftlg\" (UniqueName: \"kubernetes.io/projected/755f9627-4c80-4597-ac00-670d261bb1ba-kube-api-access-mftlg\") pod \"image-registry-f4c5cdd74-l4dbz\" (UID: \"755f9627-4c80-4597-ac00-670d261bb1ba\") " pod="openshift-image-registry/image-registry-f4c5cdd74-l4dbz"
Jul 21 12:40:59.746431 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:59.746363    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/755f9627-4c80-4597-ac00-670d261bb1ba-installation-pull-secrets\") pod \"image-registry-f4c5cdd74-l4dbz\" (UID: \"755f9627-4c80-4597-ac00-670d261bb1ba\") " pod="openshift-image-registry/image-registry-f4c5cdd74-l4dbz"
Jul 21 12:40:59.746431 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:59.746391    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/755f9627-4c80-4597-ac00-670d261bb1ba-trusted-ca\") pod \"image-registry-f4c5cdd74-l4dbz\" (UID: \"755f9627-4c80-4597-ac00-670d261bb1ba\") " pod="openshift-image-registry/image-registry-f4c5cdd74-l4dbz"
Jul 21 12:40:59.746509 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:59.746485    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/755f9627-4c80-4597-ac00-670d261bb1ba-image-registry-private-configuration\") pod \"image-registry-f4c5cdd74-l4dbz\" (UID: \"755f9627-4c80-4597-ac00-670d261bb1ba\") " pod="openshift-image-registry/image-registry-f4c5cdd74-l4dbz"
Jul 21 12:40:59.746553 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:59.746524    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/755f9627-4c80-4597-ac00-670d261bb1ba-registry-tls\") pod \"image-registry-f4c5cdd74-l4dbz\" (UID: \"755f9627-4c80-4597-ac00-670d261bb1ba\") " pod="openshift-image-registry/image-registry-f4c5cdd74-l4dbz"
Jul 21 12:40:59.746553 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:59.746541    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/755f9627-4c80-4597-ac00-670d261bb1ba-bound-sa-token\") pod \"image-registry-f4c5cdd74-l4dbz\" (UID: \"755f9627-4c80-4597-ac00-670d261bb1ba\") " pod="openshift-image-registry/image-registry-f4c5cdd74-l4dbz"
Jul 21 12:40:59.847695 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:59.847662    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/755f9627-4c80-4597-ac00-670d261bb1ba-image-registry-private-configuration\") pod \"image-registry-f4c5cdd74-l4dbz\" (UID: \"755f9627-4c80-4597-ac00-670d261bb1ba\") " pod="openshift-image-registry/image-registry-f4c5cdd74-l4dbz"
Jul 21 12:40:59.847911 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:59.847712    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/755f9627-4c80-4597-ac00-670d261bb1ba-registry-tls\") pod \"image-registry-f4c5cdd74-l4dbz\" (UID: \"755f9627-4c80-4597-ac00-670d261bb1ba\") " pod="openshift-image-registry/image-registry-f4c5cdd74-l4dbz"
Jul 21 12:40:59.847911 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:59.847739    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/755f9627-4c80-4597-ac00-670d261bb1ba-bound-sa-token\") pod \"image-registry-f4c5cdd74-l4dbz\" (UID: \"755f9627-4c80-4597-ac00-670d261bb1ba\") " pod="openshift-image-registry/image-registry-f4c5cdd74-l4dbz"
Jul 21 12:40:59.847911 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:59.847804    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/755f9627-4c80-4597-ac00-670d261bb1ba-ca-trust-extracted\") pod \"image-registry-f4c5cdd74-l4dbz\" (UID: \"755f9627-4c80-4597-ac00-670d261bb1ba\") " pod="openshift-image-registry/image-registry-f4c5cdd74-l4dbz"
Jul 21 12:40:59.847911 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:59.847843    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/755f9627-4c80-4597-ac00-670d261bb1ba-registry-certificates\") pod \"image-registry-f4c5cdd74-l4dbz\" (UID: \"755f9627-4c80-4597-ac00-670d261bb1ba\") " pod="openshift-image-registry/image-registry-f4c5cdd74-l4dbz"
Jul 21 12:40:59.847911 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:59.847876    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mftlg\" (UniqueName: \"kubernetes.io/projected/755f9627-4c80-4597-ac00-670d261bb1ba-kube-api-access-mftlg\") pod \"image-registry-f4c5cdd74-l4dbz\" (UID: \"755f9627-4c80-4597-ac00-670d261bb1ba\") " pod="openshift-image-registry/image-registry-f4c5cdd74-l4dbz"
Jul 21 12:40:59.847911 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:59.847913    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/755f9627-4c80-4597-ac00-670d261bb1ba-installation-pull-secrets\") pod \"image-registry-f4c5cdd74-l4dbz\" (UID: \"755f9627-4c80-4597-ac00-670d261bb1ba\") " pod="openshift-image-registry/image-registry-f4c5cdd74-l4dbz"
Jul 21 12:40:59.848246 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:59.847940    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/755f9627-4c80-4597-ac00-670d261bb1ba-trusted-ca\") pod \"image-registry-f4c5cdd74-l4dbz\" (UID: \"755f9627-4c80-4597-ac00-670d261bb1ba\") " pod="openshift-image-registry/image-registry-f4c5cdd74-l4dbz"
Jul 21 12:40:59.848653 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:59.848625    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/755f9627-4c80-4597-ac00-670d261bb1ba-ca-trust-extracted\") pod \"image-registry-f4c5cdd74-l4dbz\" (UID: \"755f9627-4c80-4597-ac00-670d261bb1ba\") " pod="openshift-image-registry/image-registry-f4c5cdd74-l4dbz"
Jul 21 12:40:59.849518 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:59.849493    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/755f9627-4c80-4597-ac00-670d261bb1ba-registry-certificates\") pod \"image-registry-f4c5cdd74-l4dbz\" (UID: \"755f9627-4c80-4597-ac00-670d261bb1ba\") " pod="openshift-image-registry/image-registry-f4c5cdd74-l4dbz"
Jul 21 12:40:59.849928 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:59.849903    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/755f9627-4c80-4597-ac00-670d261bb1ba-trusted-ca\") pod \"image-registry-f4c5cdd74-l4dbz\" (UID: \"755f9627-4c80-4597-ac00-670d261bb1ba\") " pod="openshift-image-registry/image-registry-f4c5cdd74-l4dbz"
Jul 21 12:40:59.850636 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:59.850617    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/755f9627-4c80-4597-ac00-670d261bb1ba-registry-tls\") pod \"image-registry-f4c5cdd74-l4dbz\" (UID: \"755f9627-4c80-4597-ac00-670d261bb1ba\") " pod="openshift-image-registry/image-registry-f4c5cdd74-l4dbz"
Jul 21 12:40:59.850973 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:59.850950    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/755f9627-4c80-4597-ac00-670d261bb1ba-image-registry-private-configuration\") pod \"image-registry-f4c5cdd74-l4dbz\" (UID: \"755f9627-4c80-4597-ac00-670d261bb1ba\") " pod="openshift-image-registry/image-registry-f4c5cdd74-l4dbz"
Jul 21 12:40:59.851759 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:59.851737    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/755f9627-4c80-4597-ac00-670d261bb1ba-installation-pull-secrets\") pod \"image-registry-f4c5cdd74-l4dbz\" (UID: \"755f9627-4c80-4597-ac00-670d261bb1ba\") " pod="openshift-image-registry/image-registry-f4c5cdd74-l4dbz"
Jul 21 12:40:59.862051 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:59.862027    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mftlg\" (UniqueName: \"kubernetes.io/projected/755f9627-4c80-4597-ac00-670d261bb1ba-kube-api-access-mftlg\") pod \"image-registry-f4c5cdd74-l4dbz\" (UID: \"755f9627-4c80-4597-ac00-670d261bb1ba\") " pod="openshift-image-registry/image-registry-f4c5cdd74-l4dbz"
Jul 21 12:40:59.862589 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:59.862572    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/755f9627-4c80-4597-ac00-670d261bb1ba-bound-sa-token\") pod \"image-registry-f4c5cdd74-l4dbz\" (UID: \"755f9627-4c80-4597-ac00-670d261bb1ba\") " pod="openshift-image-registry/image-registry-f4c5cdd74-l4dbz"
Jul 21 12:40:59.899881 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:40:59.899831    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-f4c5cdd74-l4dbz"
Jul 21 12:41:01.302842 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:01.302815    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-f4c5cdd74-l4dbz"]
Jul 21 12:41:01.313138 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:41:01.313112    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod755f9627_4c80_4597_ac00_670d261bb1ba.slice/crio-316adc1a6f91a8429cb0b39848235432ffdcb74051fa0510794325a83f07ee0d WatchSource:0}: Error finding container 316adc1a6f91a8429cb0b39848235432ffdcb74051fa0510794325a83f07ee0d: Status 404 returned error can't find the container with id 316adc1a6f91a8429cb0b39848235432ffdcb74051fa0510794325a83f07ee0d
Jul 21 12:41:01.330737 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:01.330683    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-765f54f9d6-t2xgw"]
Jul 21 12:41:01.518599 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:01.518563    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-clxmp" event={"ID":"37f02e4b-2a9f-4e44-8c74-d2428bd97ee5","Type":"ContainerStarted","Data":"c46c295e24405637ad1076e37dfe7277988b4f0d9df11e1ef88044b7a967725b"}
Jul 21 12:41:01.518766 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:01.518658    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-clxmp"
Jul 21 12:41:01.520242 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:01.520215    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-mchwp" event={"ID":"d71014d4-ae0b-437e-adca-886a8ff5ac7f","Type":"ContainerStarted","Data":"8632f7cee86b3b42a838e15210f34938cd05ffaf55694f469c074ba1fcc1f432"}
Jul 21 12:41:01.522470 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:01.522446    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-94clr" event={"ID":"eb9d39d0-f6a2-4050-a227-8aa6061b4799","Type":"ContainerStarted","Data":"7a7dd6f08c95e58f84e311f09683e2056190f17019f8d378d877892a059ed9dc"}
Jul 21 12:41:01.522564 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:01.522494    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-94clr" event={"ID":"eb9d39d0-f6a2-4050-a227-8aa6061b4799","Type":"ContainerStarted","Data":"46306e25e5408582da6e4fc08bf333eab50d62bd4b0c3fb0ed8b0ec222fa09e4"}
Jul 21 12:41:01.522932 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:01.522917    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-94clr"
Jul 21 12:41:01.524813 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:01.524791    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-f4c5cdd74-l4dbz" event={"ID":"755f9627-4c80-4597-ac00-670d261bb1ba","Type":"ContainerStarted","Data":"881e37add7c08d644eb890e2bea37ff086dad0c9f0332273efdede2a652035d5"}
Jul 21 12:41:01.524922 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:01.524829    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-f4c5cdd74-l4dbz" event={"ID":"755f9627-4c80-4597-ac00-670d261bb1ba","Type":"ContainerStarted","Data":"316adc1a6f91a8429cb0b39848235432ffdcb74051fa0510794325a83f07ee0d"}
Jul 21 12:41:01.525053 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:01.525032    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-f4c5cdd74-l4dbz"
Jul 21 12:41:01.528557 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:01.527167    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-cldzr" event={"ID":"a087fecd-1bf9-4449-a9a1-dc1aba2a2ec3","Type":"ContainerStarted","Data":"37289a1fad01f3cb11f5f236bdba9a82a52a970ecdea2cf97a620f6bc5bfbb48"}
Jul 21 12:41:01.529831 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:01.529744    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-lrn59" event={"ID":"9bf9a989-c13b-4419-84f7-ddab7a99fedf","Type":"ContainerStarted","Data":"2d0ce0d034829f31ec5f8d2d426642bc46141b7f8ac9e15968cbefe54f1eb621"}
Jul 21 12:41:01.529831 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:01.529797    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-lrn59" event={"ID":"9bf9a989-c13b-4419-84f7-ddab7a99fedf","Type":"ContainerStarted","Data":"731c0e8144c6bae7b158786d57686c88a201d8598eeeadb063c7ea4457a6144f"}
Jul 21 12:41:01.532735 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:01.532704    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-765f54f9d6-t2xgw" event={"ID":"38f8b753-80af-418e-b400-ebfe518a3a71","Type":"ContainerStarted","Data":"a4ccfea04b3eb49f85c413204e1734f32699700516aa7deccfe2c5d48c141718"}
Jul 21 12:41:01.539321 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:01.539283    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-clxmp" podStartSLOduration=66.769662987 podStartE2EDuration="1m14.539270857s" totalImagesPullingTime="7.76960787s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:39:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:41:01.538424938 +0000 UTC m=+74.842430028" watchObservedRunningTime="2026-07-21 12:41:01.539270857 +0000 UTC m=+74.843275946"
Jul 21 12:41:01.559455 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:01.559419    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-mchwp" podStartSLOduration=65.800673159 podStartE2EDuration="1m13.559403853s" totalImagesPullingTime="7.758730694s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 12:39:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:41:01.558280055 +0000 UTC m=+74.862285146" watchObservedRunningTime="2026-07-21 12:41:01.559403853 +0000 UTC m=+74.863408920"
Jul 21 12:41:01.576420 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:01.576371    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-94clr" podStartSLOduration=33.029584854 podStartE2EDuration="39.576352222s" totalImagesPullingTime="6.546767368s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:40:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:41:01.575815252 +0000 UTC m=+74.879820344" watchObservedRunningTime="2026-07-21 12:41:01.576352222 +0000 UTC m=+74.880357313"
Jul 21 12:41:01.595191 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:01.595096    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-cldzr" podStartSLOduration=33.498992365 podStartE2EDuration="39.595073813s" totalImagesPullingTime="6.096081448s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:40:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:41:01.593580002 +0000 UTC m=+74.897585123" watchObservedRunningTime="2026-07-21 12:41:01.595073813 +0000 UTC m=+74.899078883"
Jul 21 12:41:01.616026 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:01.615951    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-f4c5cdd74-l4dbz" podStartSLOduration=2.615931864 podStartE2EDuration="2.615931864s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:40:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:41:01.615028153 +0000 UTC m=+74.919033263" watchObservedRunningTime="2026-07-21 12:41:01.615931864 +0000 UTC m=+74.919936957"
Jul 21 12:41:01.632757 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:01.632715    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-lrn59" podStartSLOduration=33.605824614 podStartE2EDuration="39.632702071s" totalImagesPullingTime="6.026877457s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:40:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:41:01.631416951 +0000 UTC m=+74.935422062" watchObservedRunningTime="2026-07-21 12:41:01.632702071 +0000 UTC m=+74.936707164"
Jul 21 12:41:02.570032 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:02.569985    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/dfe654b2-569e-4c88-a935-0f82db6839e5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9j6f5\" (UID: \"dfe654b2-569e-4c88-a935-0f82db6839e5\") " pod="openshift-insights/insights-runtime-extractor-9j6f5"
Jul 21 12:41:02.572540 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:02.572518    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/dfe654b2-569e-4c88-a935-0f82db6839e5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9j6f5\" (UID: \"dfe654b2-569e-4c88-a935-0f82db6839e5\") " pod="openshift-insights/insights-runtime-extractor-9j6f5"
Jul 21 12:41:02.580061 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:02.580036    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-6mtb2\""
Jul 21 12:41:02.587949 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:02.587934    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-9j6f5"
Jul 21 12:41:02.718126 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:02.718099    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-9j6f5"]
Jul 21 12:41:02.720832 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:41:02.720802    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddfe654b2_569e_4c88_a935_0f82db6839e5.slice/crio-a85f1e7f206a3cbacd7f33c557c9c0168273d493b15fadf92da4db630aeb2fc6 WatchSource:0}: Error finding container a85f1e7f206a3cbacd7f33c557c9c0168273d493b15fadf92da4db630aeb2fc6: Status 404 returned error can't find the container with id a85f1e7f206a3cbacd7f33c557c9c0168273d493b15fadf92da4db630aeb2fc6
Jul 21 12:41:03.542286 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:03.542203    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-9j6f5" event={"ID":"dfe654b2-569e-4c88-a935-0f82db6839e5","Type":"ContainerStarted","Data":"a85f1e7f206a3cbacd7f33c557c9c0168273d493b15fadf92da4db630aeb2fc6"}
Jul 21 12:41:08.568584 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:08.568452    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-9j6f5" event={"ID":"dfe654b2-569e-4c88-a935-0f82db6839e5","Type":"ContainerStarted","Data":"840d519e4b66a8adf2a169ac3983a0057e3188d56a340c91deb9b26fbd85f4ba"}
Jul 21 12:41:11.581122 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:11.581079    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-765f54f9d6-t2xgw" event={"ID":"38f8b753-80af-418e-b400-ebfe518a3a71","Type":"ContainerStarted","Data":"35b2d91dc53389830efb9e53794da6f1725bdf1cb721ab77874889a7da49a2cb"}
Jul 21 12:41:11.601254 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:11.601211    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-765f54f9d6-t2xgw" podStartSLOduration=2.435177396 podStartE2EDuration="12.601195736s" totalImagesPullingTime="10.16601834s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:40:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:41:11.600091538 +0000 UTC m=+84.904096629" watchObservedRunningTime="2026-07-21 12:41:11.601195736 +0000 UTC m=+84.905200817"
Jul 21 12:41:12.544439 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:12.544412    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-94clr"
Jul 21 12:41:12.586132 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:12.586097    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-9j6f5" event={"ID":"dfe654b2-569e-4c88-a935-0f82db6839e5","Type":"ContainerStarted","Data":"41607c38e1cdd10bec30a5c477cc569e5e07bfa65481c61586aef11dd2d1015a"}
Jul 21 12:41:15.301823 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:15.301779    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-j744v"]
Jul 21 12:41:15.315689 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:15.315656    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56c5594674-j744v"
Jul 21 12:41:15.319309 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:15.319285    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 21 12:41:15.320654 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:15.319310    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Jul 21 12:41:15.320654 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:15.320603    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-7qz8f\""
Jul 21 12:41:15.320654 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:15.320619    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Jul 21 12:41:15.320906 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:15.320865    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Jul 21 12:41:15.324130 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:15.322611    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-j6xgk"]
Jul 21 12:41:15.327115 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:15.326464    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-j744v"]
Jul 21 12:41:15.327115 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:15.326565    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-j6xgk"
Jul 21 12:41:15.330174 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:15.329149    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 21 12:41:15.330174 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:15.329539    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-nj2hc\""
Jul 21 12:41:15.330174 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:15.329738    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 21 12:41:15.330174 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:15.329946    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 21 12:41:15.477447 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:15.477411    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/537b3296-9a40-42c6-9e84-456cc7f20f28-node-exporter-textfile\") pod \"node-exporter-j6xgk\" (UID: \"537b3296-9a40-42c6-9e84-456cc7f20f28\") " pod="openshift-monitoring/node-exporter-j6xgk"
Jul 21 12:41:15.477624 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:15.477464    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/537b3296-9a40-42c6-9e84-456cc7f20f28-node-exporter-wtmp\") pod \"node-exporter-j6xgk\" (UID: \"537b3296-9a40-42c6-9e84-456cc7f20f28\") " pod="openshift-monitoring/node-exporter-j6xgk"
Jul 21 12:41:15.477624 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:15.477491    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/537b3296-9a40-42c6-9e84-456cc7f20f28-sys\") pod \"node-exporter-j6xgk\" (UID: \"537b3296-9a40-42c6-9e84-456cc7f20f28\") " pod="openshift-monitoring/node-exporter-j6xgk"
Jul 21 12:41:15.477624 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:15.477536    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/eaaabf68-0d82-4671-a030-fead13f69633-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-j744v\" (UID: \"eaaabf68-0d82-4671-a030-fead13f69633\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-j744v"
Jul 21 12:41:15.477755 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:15.477635    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/eaaabf68-0d82-4671-a030-fead13f69633-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-j744v\" (UID: \"eaaabf68-0d82-4671-a030-fead13f69633\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-j744v"
Jul 21 12:41:15.477755 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:15.477681    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/537b3296-9a40-42c6-9e84-456cc7f20f28-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-j6xgk\" (UID: \"537b3296-9a40-42c6-9e84-456cc7f20f28\") " pod="openshift-monitoring/node-exporter-j6xgk"
Jul 21 12:41:15.477755 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:15.477720    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q76np\" (UniqueName: \"kubernetes.io/projected/eaaabf68-0d82-4671-a030-fead13f69633-kube-api-access-q76np\") pod \"kube-state-metrics-56c5594674-j744v\" (UID: \"eaaabf68-0d82-4671-a030-fead13f69633\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-j744v"
Jul 21 12:41:15.477886 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:15.477759    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/eaaabf68-0d82-4671-a030-fead13f69633-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-j744v\" (UID: \"eaaabf68-0d82-4671-a030-fead13f69633\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-j744v"
Jul 21 12:41:15.477886 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:15.477792    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/537b3296-9a40-42c6-9e84-456cc7f20f28-root\") pod \"node-exporter-j6xgk\" (UID: \"537b3296-9a40-42c6-9e84-456cc7f20f28\") " pod="openshift-monitoring/node-exporter-j6xgk"
Jul 21 12:41:15.477886 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:15.477819    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/eaaabf68-0d82-4671-a030-fead13f69633-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-j744v\" (UID: \"eaaabf68-0d82-4671-a030-fead13f69633\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-j744v"
Jul 21 12:41:15.477886 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:15.477845    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/537b3296-9a40-42c6-9e84-456cc7f20f28-metrics-client-ca\") pod \"node-exporter-j6xgk\" (UID: \"537b3296-9a40-42c6-9e84-456cc7f20f28\") " pod="openshift-monitoring/node-exporter-j6xgk"
Jul 21 12:41:15.478098 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:15.477906    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fnf5r\" (UniqueName: \"kubernetes.io/projected/537b3296-9a40-42c6-9e84-456cc7f20f28-kube-api-access-fnf5r\") pod \"node-exporter-j6xgk\" (UID: \"537b3296-9a40-42c6-9e84-456cc7f20f28\") " pod="openshift-monitoring/node-exporter-j6xgk"
Jul 21 12:41:15.478098 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:15.477949    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/537b3296-9a40-42c6-9e84-456cc7f20f28-node-exporter-tls\") pod \"node-exporter-j6xgk\" (UID: \"537b3296-9a40-42c6-9e84-456cc7f20f28\") " pod="openshift-monitoring/node-exporter-j6xgk"
Jul 21 12:41:15.478098 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:15.477981    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/537b3296-9a40-42c6-9e84-456cc7f20f28-node-exporter-accelerators-collector-config\") pod \"node-exporter-j6xgk\" (UID: \"537b3296-9a40-42c6-9e84-456cc7f20f28\") " pod="openshift-monitoring/node-exporter-j6xgk"
Jul 21 12:41:15.478098 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:15.478079    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/eaaabf68-0d82-4671-a030-fead13f69633-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-j744v\" (UID: \"eaaabf68-0d82-4671-a030-fead13f69633\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-j744v"
Jul 21 12:41:15.578930 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:15.578853    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/eaaabf68-0d82-4671-a030-fead13f69633-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-j744v\" (UID: \"eaaabf68-0d82-4671-a030-fead13f69633\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-j744v"
Jul 21 12:41:15.578930 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:15.578903    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/537b3296-9a40-42c6-9e84-456cc7f20f28-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-j6xgk\" (UID: \"537b3296-9a40-42c6-9e84-456cc7f20f28\") " pod="openshift-monitoring/node-exporter-j6xgk"
Jul 21 12:41:15.579172 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:15.578937    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q76np\" (UniqueName: \"kubernetes.io/projected/eaaabf68-0d82-4671-a030-fead13f69633-kube-api-access-q76np\") pod \"kube-state-metrics-56c5594674-j744v\" (UID: \"eaaabf68-0d82-4671-a030-fead13f69633\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-j744v"
Jul 21 12:41:15.579172 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:15.578974    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/eaaabf68-0d82-4671-a030-fead13f69633-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-j744v\" (UID: \"eaaabf68-0d82-4671-a030-fead13f69633\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-j744v"
Jul 21 12:41:15.579172 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:15.579005    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/537b3296-9a40-42c6-9e84-456cc7f20f28-root\") pod \"node-exporter-j6xgk\" (UID: \"537b3296-9a40-42c6-9e84-456cc7f20f28\") " pod="openshift-monitoring/node-exporter-j6xgk"
Jul 21 12:41:15.579172 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:15.579060    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/eaaabf68-0d82-4671-a030-fead13f69633-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-j744v\" (UID: \"eaaabf68-0d82-4671-a030-fead13f69633\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-j744v"
Jul 21 12:41:15.579172 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:15.579088    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/537b3296-9a40-42c6-9e84-456cc7f20f28-metrics-client-ca\") pod \"node-exporter-j6xgk\" (UID: \"537b3296-9a40-42c6-9e84-456cc7f20f28\") " pod="openshift-monitoring/node-exporter-j6xgk"
Jul 21 12:41:15.579172 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:15.579118    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fnf5r\" (UniqueName: \"kubernetes.io/projected/537b3296-9a40-42c6-9e84-456cc7f20f28-kube-api-access-fnf5r\") pod \"node-exporter-j6xgk\" (UID: \"537b3296-9a40-42c6-9e84-456cc7f20f28\") " pod="openshift-monitoring/node-exporter-j6xgk"
Jul 21 12:41:15.579172 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:15.579147    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/537b3296-9a40-42c6-9e84-456cc7f20f28-node-exporter-tls\") pod \"node-exporter-j6xgk\" (UID: \"537b3296-9a40-42c6-9e84-456cc7f20f28\") " pod="openshift-monitoring/node-exporter-j6xgk"
Jul 21 12:41:15.579172 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:15.579173    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/537b3296-9a40-42c6-9e84-456cc7f20f28-node-exporter-accelerators-collector-config\") pod \"node-exporter-j6xgk\" (UID: \"537b3296-9a40-42c6-9e84-456cc7f20f28\") " pod="openshift-monitoring/node-exporter-j6xgk"
Jul 21 12:41:15.579480 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:15.579215    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/eaaabf68-0d82-4671-a030-fead13f69633-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-j744v\" (UID: \"eaaabf68-0d82-4671-a030-fead13f69633\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-j744v"
Jul 21 12:41:15.579480 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:15.579247    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/537b3296-9a40-42c6-9e84-456cc7f20f28-node-exporter-textfile\") pod \"node-exporter-j6xgk\" (UID: \"537b3296-9a40-42c6-9e84-456cc7f20f28\") " pod="openshift-monitoring/node-exporter-j6xgk"
Jul 21 12:41:15.579480 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:15.579265    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/537b3296-9a40-42c6-9e84-456cc7f20f28-node-exporter-wtmp\") pod \"node-exporter-j6xgk\" (UID: \"537b3296-9a40-42c6-9e84-456cc7f20f28\") " pod="openshift-monitoring/node-exporter-j6xgk"
Jul 21 12:41:15.579480 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:15.579289    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/537b3296-9a40-42c6-9e84-456cc7f20f28-sys\") pod \"node-exporter-j6xgk\" (UID: \"537b3296-9a40-42c6-9e84-456cc7f20f28\") " pod="openshift-monitoring/node-exporter-j6xgk"
Jul 21 12:41:15.579480 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:15.579338    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/eaaabf68-0d82-4671-a030-fead13f69633-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-j744v\" (UID: \"eaaabf68-0d82-4671-a030-fead13f69633\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-j744v"
Jul 21 12:41:15.579881 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:15.579859    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/537b3296-9a40-42c6-9e84-456cc7f20f28-metrics-client-ca\") pod \"node-exporter-j6xgk\" (UID: \"537b3296-9a40-42c6-9e84-456cc7f20f28\") " pod="openshift-monitoring/node-exporter-j6xgk"
Jul 21 12:41:15.579963 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:15.579903    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/eaaabf68-0d82-4671-a030-fead13f69633-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-j744v\" (UID: \"eaaabf68-0d82-4671-a030-fead13f69633\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-j744v"
Jul 21 12:41:15.581307 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:15.580364    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/eaaabf68-0d82-4671-a030-fead13f69633-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-j744v\" (UID: \"eaaabf68-0d82-4671-a030-fead13f69633\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-j744v"
Jul 21 12:41:15.581307 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:15.580484    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/537b3296-9a40-42c6-9e84-456cc7f20f28-root\") pod \"node-exporter-j6xgk\" (UID: \"537b3296-9a40-42c6-9e84-456cc7f20f28\") " pod="openshift-monitoring/node-exporter-j6xgk"
Jul 21 12:41:15.581307 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:15.580603    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/537b3296-9a40-42c6-9e84-456cc7f20f28-node-exporter-textfile\") pod \"node-exporter-j6xgk\" (UID: \"537b3296-9a40-42c6-9e84-456cc7f20f28\") " pod="openshift-monitoring/node-exporter-j6xgk"
Jul 21 12:41:15.581307 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:15.580735    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/537b3296-9a40-42c6-9e84-456cc7f20f28-node-exporter-wtmp\") pod \"node-exporter-j6xgk\" (UID: \"537b3296-9a40-42c6-9e84-456cc7f20f28\") " pod="openshift-monitoring/node-exporter-j6xgk"
Jul 21 12:41:15.581307 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:15.580784    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/537b3296-9a40-42c6-9e84-456cc7f20f28-sys\") pod \"node-exporter-j6xgk\" (UID: \"537b3296-9a40-42c6-9e84-456cc7f20f28\") " pod="openshift-monitoring/node-exporter-j6xgk"
Jul 21 12:41:15.581307 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:15.581113    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/eaaabf68-0d82-4671-a030-fead13f69633-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-j744v\" (UID: \"eaaabf68-0d82-4671-a030-fead13f69633\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-j744v"
Jul 21 12:41:15.581307 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:15.581269    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/537b3296-9a40-42c6-9e84-456cc7f20f28-node-exporter-accelerators-collector-config\") pod \"node-exporter-j6xgk\" (UID: \"537b3296-9a40-42c6-9e84-456cc7f20f28\") " pod="openshift-monitoring/node-exporter-j6xgk"
Jul 21 12:41:15.582071 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:15.581999    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/537b3296-9a40-42c6-9e84-456cc7f20f28-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-j6xgk\" (UID: \"537b3296-9a40-42c6-9e84-456cc7f20f28\") " pod="openshift-monitoring/node-exporter-j6xgk"
Jul 21 12:41:15.582892 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:15.582872    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/eaaabf68-0d82-4671-a030-fead13f69633-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-j744v\" (UID: \"eaaabf68-0d82-4671-a030-fead13f69633\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-j744v"
Jul 21 12:41:15.583211 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:15.583185    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/eaaabf68-0d82-4671-a030-fead13f69633-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-j744v\" (UID: \"eaaabf68-0d82-4671-a030-fead13f69633\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-j744v"
Jul 21 12:41:15.583624 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:15.583603    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/537b3296-9a40-42c6-9e84-456cc7f20f28-node-exporter-tls\") pod \"node-exporter-j6xgk\" (UID: \"537b3296-9a40-42c6-9e84-456cc7f20f28\") " pod="openshift-monitoring/node-exporter-j6xgk"
Jul 21 12:41:15.590545 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:15.590341    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fnf5r\" (UniqueName: \"kubernetes.io/projected/537b3296-9a40-42c6-9e84-456cc7f20f28-kube-api-access-fnf5r\") pod \"node-exporter-j6xgk\" (UID: \"537b3296-9a40-42c6-9e84-456cc7f20f28\") " pod="openshift-monitoring/node-exporter-j6xgk"
Jul 21 12:41:15.594650 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:15.593092    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q76np\" (UniqueName: \"kubernetes.io/projected/eaaabf68-0d82-4671-a030-fead13f69633-kube-api-access-q76np\") pod \"kube-state-metrics-56c5594674-j744v\" (UID: \"eaaabf68-0d82-4671-a030-fead13f69633\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-j744v"
Jul 21 12:41:15.632119 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:15.632080    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56c5594674-j744v"
Jul 21 12:41:15.640212 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:15.640189    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-j6xgk"
Jul 21 12:41:16.214087 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:41:16.214056    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod537b3296_9a40_42c6_9e84_456cc7f20f28.slice/crio-59012bfbc4d03919a3627263018dc3710ae1d353ed6b778d1dd31527379c211e WatchSource:0}: Error finding container 59012bfbc4d03919a3627263018dc3710ae1d353ed6b778d1dd31527379c211e: Status 404 returned error can't find the container with id 59012bfbc4d03919a3627263018dc3710ae1d353ed6b778d1dd31527379c211e
Jul 21 12:41:16.356920 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:16.356828    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-j744v"]
Jul 21 12:41:16.359307 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:41:16.359282    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeaaabf68_0d82_4671_a030_fead13f69633.slice/crio-4cb10bde9d1c49446e8c29a544adfb1cc137096fd3a1432b1c9c3d0266d8c11a WatchSource:0}: Error finding container 4cb10bde9d1c49446e8c29a544adfb1cc137096fd3a1432b1c9c3d0266d8c11a: Status 404 returned error can't find the container with id 4cb10bde9d1c49446e8c29a544adfb1cc137096fd3a1432b1c9c3d0266d8c11a
Jul 21 12:41:16.433049 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:16.433005    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 21 12:41:16.436819 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:16.436805    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:41:16.439614 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:16.439589    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-kbkl6\""
Jul 21 12:41:16.439614 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:16.439608    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 21 12:41:16.439764 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:16.439596    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 21 12:41:16.439764 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:16.439609    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 21 12:41:16.439866 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:16.439787    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 21 12:41:16.440049 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:16.440030    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 21 12:41:16.440134 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:16.440051    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 21 12:41:16.440134 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:16.440035    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 21 12:41:16.440358 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:16.440345    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 21 12:41:16.445267 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:16.445242    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 21 12:41:16.469732 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:16.469682    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 21 12:41:16.586303 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:16.586278    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/7438cb69-67ff-4740-aa8f-8501300d0604-config-volume\") pod \"alertmanager-main-0\" (UID: \"7438cb69-67ff-4740-aa8f-8501300d0604\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:41:16.586413 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:16.586313    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7438cb69-67ff-4740-aa8f-8501300d0604-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"7438cb69-67ff-4740-aa8f-8501300d0604\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:41:16.586413 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:16.586342    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7438cb69-67ff-4740-aa8f-8501300d0604-web-config\") pod \"alertmanager-main-0\" (UID: \"7438cb69-67ff-4740-aa8f-8501300d0604\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:41:16.586413 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:16.586402    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/7438cb69-67ff-4740-aa8f-8501300d0604-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"7438cb69-67ff-4740-aa8f-8501300d0604\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:41:16.586526 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:16.586456    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7438cb69-67ff-4740-aa8f-8501300d0604-config-out\") pod \"alertmanager-main-0\" (UID: \"7438cb69-67ff-4740-aa8f-8501300d0604\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:41:16.586526 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:16.586495    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7438cb69-67ff-4740-aa8f-8501300d0604-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"7438cb69-67ff-4740-aa8f-8501300d0604\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:41:16.586612 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:16.586528    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7438cb69-67ff-4740-aa8f-8501300d0604-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"7438cb69-67ff-4740-aa8f-8501300d0604\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:41:16.586612 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:16.586561    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7438cb69-67ff-4740-aa8f-8501300d0604-tls-assets\") pod \"alertmanager-main-0\" (UID: \"7438cb69-67ff-4740-aa8f-8501300d0604\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:41:16.586612 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:16.586590    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7438cb69-67ff-4740-aa8f-8501300d0604-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"7438cb69-67ff-4740-aa8f-8501300d0604\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:41:16.586740 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:16.586640    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/7438cb69-67ff-4740-aa8f-8501300d0604-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"7438cb69-67ff-4740-aa8f-8501300d0604\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:41:16.586740 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:16.586692    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/7438cb69-67ff-4740-aa8f-8501300d0604-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"7438cb69-67ff-4740-aa8f-8501300d0604\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:41:16.586740 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:16.586719    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/7438cb69-67ff-4740-aa8f-8501300d0604-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"7438cb69-67ff-4740-aa8f-8501300d0604\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:41:16.586861 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:16.586748    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-872jt\" (UniqueName: \"kubernetes.io/projected/7438cb69-67ff-4740-aa8f-8501300d0604-kube-api-access-872jt\") pod \"alertmanager-main-0\" (UID: \"7438cb69-67ff-4740-aa8f-8501300d0604\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:41:16.603241 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:16.603217    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-9j6f5" event={"ID":"dfe654b2-569e-4c88-a935-0f82db6839e5","Type":"ContainerStarted","Data":"9f5241ab224736d6d57ba5192613f0a47c2cb23615044ba820da0a4009fb2ee2"}
Jul 21 12:41:16.604462 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:16.604442    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-j744v" event={"ID":"eaaabf68-0d82-4671-a030-fead13f69633","Type":"ContainerStarted","Data":"4cb10bde9d1c49446e8c29a544adfb1cc137096fd3a1432b1c9c3d0266d8c11a"}
Jul 21 12:41:16.605527 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:16.605508    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-j6xgk" event={"ID":"537b3296-9a40-42c6-9e84-456cc7f20f28","Type":"ContainerStarted","Data":"59012bfbc4d03919a3627263018dc3710ae1d353ed6b778d1dd31527379c211e"}
Jul 21 12:41:16.621960 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:16.621922    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-9j6f5" podStartSLOduration=33.197419812 podStartE2EDuration="46.621909726s" totalImagesPullingTime="13.424489914s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 12:40:30 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:41:16.621070355 +0000 UTC m=+89.925075463" watchObservedRunningTime="2026-07-21 12:41:16.621909726 +0000 UTC m=+89.925914816"
Jul 21 12:41:16.687406 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:16.687381    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7438cb69-67ff-4740-aa8f-8501300d0604-config-out\") pod \"alertmanager-main-0\" (UID: \"7438cb69-67ff-4740-aa8f-8501300d0604\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:41:16.687507 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:16.687420    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7438cb69-67ff-4740-aa8f-8501300d0604-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"7438cb69-67ff-4740-aa8f-8501300d0604\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:41:16.687507 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:16.687447    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7438cb69-67ff-4740-aa8f-8501300d0604-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"7438cb69-67ff-4740-aa8f-8501300d0604\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:41:16.687507 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:16.687476    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7438cb69-67ff-4740-aa8f-8501300d0604-tls-assets\") pod \"alertmanager-main-0\" (UID: \"7438cb69-67ff-4740-aa8f-8501300d0604\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:41:16.687507 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:16.687501    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7438cb69-67ff-4740-aa8f-8501300d0604-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"7438cb69-67ff-4740-aa8f-8501300d0604\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:41:16.687709 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:16.687537    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/7438cb69-67ff-4740-aa8f-8501300d0604-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"7438cb69-67ff-4740-aa8f-8501300d0604\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:41:16.687709 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:16.687579    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/7438cb69-67ff-4740-aa8f-8501300d0604-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"7438cb69-67ff-4740-aa8f-8501300d0604\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:41:16.688198 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:16.687855    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/7438cb69-67ff-4740-aa8f-8501300d0604-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"7438cb69-67ff-4740-aa8f-8501300d0604\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:41:16.688198 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:16.687899    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-872jt\" (UniqueName: \"kubernetes.io/projected/7438cb69-67ff-4740-aa8f-8501300d0604-kube-api-access-872jt\") pod \"alertmanager-main-0\" (UID: \"7438cb69-67ff-4740-aa8f-8501300d0604\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:41:16.688198 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:16.687932    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/7438cb69-67ff-4740-aa8f-8501300d0604-config-volume\") pod \"alertmanager-main-0\" (UID: \"7438cb69-67ff-4740-aa8f-8501300d0604\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:41:16.688198 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:16.687970    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7438cb69-67ff-4740-aa8f-8501300d0604-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"7438cb69-67ff-4740-aa8f-8501300d0604\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:41:16.688198 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:16.688001    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7438cb69-67ff-4740-aa8f-8501300d0604-web-config\") pod \"alertmanager-main-0\" (UID: \"7438cb69-67ff-4740-aa8f-8501300d0604\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:41:16.688198 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:16.688058    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/7438cb69-67ff-4740-aa8f-8501300d0604-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"7438cb69-67ff-4740-aa8f-8501300d0604\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:41:16.688198 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:16.688073    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/7438cb69-67ff-4740-aa8f-8501300d0604-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"7438cb69-67ff-4740-aa8f-8501300d0604\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:41:16.688909 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:16.688247    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7438cb69-67ff-4740-aa8f-8501300d0604-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"7438cb69-67ff-4740-aa8f-8501300d0604\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:41:16.688909 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:16.688489    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7438cb69-67ff-4740-aa8f-8501300d0604-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"7438cb69-67ff-4740-aa8f-8501300d0604\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:41:16.690931 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:16.690903    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7438cb69-67ff-4740-aa8f-8501300d0604-tls-assets\") pod \"alertmanager-main-0\" (UID: \"7438cb69-67ff-4740-aa8f-8501300d0604\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:41:16.691371 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:16.691347    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/7438cb69-67ff-4740-aa8f-8501300d0604-config-volume\") pod \"alertmanager-main-0\" (UID: \"7438cb69-67ff-4740-aa8f-8501300d0604\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:41:16.691967 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:16.691809    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/7438cb69-67ff-4740-aa8f-8501300d0604-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"7438cb69-67ff-4740-aa8f-8501300d0604\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:41:16.691967 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:16.691926    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/7438cb69-67ff-4740-aa8f-8501300d0604-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"7438cb69-67ff-4740-aa8f-8501300d0604\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:41:16.691967 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:16.691939    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7438cb69-67ff-4740-aa8f-8501300d0604-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"7438cb69-67ff-4740-aa8f-8501300d0604\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:41:16.692591 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:16.692565    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7438cb69-67ff-4740-aa8f-8501300d0604-config-out\") pod \"alertmanager-main-0\" (UID: \"7438cb69-67ff-4740-aa8f-8501300d0604\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:41:16.692692 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:16.692673    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7438cb69-67ff-4740-aa8f-8501300d0604-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"7438cb69-67ff-4740-aa8f-8501300d0604\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:41:16.692735 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:16.692698    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/7438cb69-67ff-4740-aa8f-8501300d0604-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"7438cb69-67ff-4740-aa8f-8501300d0604\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:41:16.692890 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:16.692874    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7438cb69-67ff-4740-aa8f-8501300d0604-web-config\") pod \"alertmanager-main-0\" (UID: \"7438cb69-67ff-4740-aa8f-8501300d0604\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:41:16.697298 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:16.697274    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-872jt\" (UniqueName: \"kubernetes.io/projected/7438cb69-67ff-4740-aa8f-8501300d0604-kube-api-access-872jt\") pod \"alertmanager-main-0\" (UID: \"7438cb69-67ff-4740-aa8f-8501300d0604\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:41:16.746141 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:16.746093    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:41:16.912838 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:16.912809    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 21 12:41:16.916507 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:41:16.916479    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7438cb69_67ff_4740_aa8f_8501300d0604.slice/crio-eec9a7200f3e236744492a1b3f93757c3f5915be3feb0ff8903a8716ce3d539a WatchSource:0}: Error finding container eec9a7200f3e236744492a1b3f93757c3f5915be3feb0ff8903a8716ce3d539a: Status 404 returned error can't find the container with id eec9a7200f3e236744492a1b3f93757c3f5915be3feb0ff8903a8716ce3d539a
Jul 21 12:41:17.610510 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:17.610440    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7438cb69-67ff-4740-aa8f-8501300d0604","Type":"ContainerStarted","Data":"eec9a7200f3e236744492a1b3f93757c3f5915be3feb0ff8903a8716ce3d539a"}
Jul 21 12:41:17.612007 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:17.611979    2556 generic.go:356] "Generic (PLEG): container finished" podID="537b3296-9a40-42c6-9e84-456cc7f20f28" containerID="3166466459ed1121d4bdb9db372f73bf8a4803f2d359c67edf5ebcec122b51bc" exitCode=0
Jul 21 12:41:17.612142 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:17.612045    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-j6xgk" event={"ID":"537b3296-9a40-42c6-9e84-456cc7f20f28","Type":"ContainerDied","Data":"3166466459ed1121d4bdb9db372f73bf8a4803f2d359c67edf5ebcec122b51bc"}
Jul 21 12:41:18.618914 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:18.618881    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-j6xgk" event={"ID":"537b3296-9a40-42c6-9e84-456cc7f20f28","Type":"ContainerStarted","Data":"bbb3fb01c2f6166244806a2aebbafa43a226c8ddd1127750d6dd83b24d67df81"}
Jul 21 12:41:18.619401 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:18.618946    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-j6xgk" event={"ID":"537b3296-9a40-42c6-9e84-456cc7f20f28","Type":"ContainerStarted","Data":"d07c37eeef098ce333d651bb359211d2b95e11197a01bccf0e4d36a1b80afa5a"}
Jul 21 12:41:18.641214 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:18.641160    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-j6xgk" podStartSLOduration=2.555989239 podStartE2EDuration="3.641144766s" totalImagesPullingTime="1.014432477s" totalInitContainerRuntime="70.72305ms" isStatefulPod=true podCreationTimestamp="2026-07-21 12:41:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:41:18.639575403 +0000 UTC m=+91.943580493" watchObservedRunningTime="2026-07-21 12:41:18.641144766 +0000 UTC m=+91.945149856"
Jul 21 12:41:19.448527 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:19.448502    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-5cffd5944d-w72zg"
Jul 21 12:41:19.625739 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:19.625708    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-j744v" event={"ID":"eaaabf68-0d82-4671-a030-fead13f69633","Type":"ContainerStarted","Data":"f516a8bbcff1ec159ec5f243261b034f385f47d235ed01884cf27ff7f0774617"}
Jul 21 12:41:19.626218 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:19.625768    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-j744v" event={"ID":"eaaabf68-0d82-4671-a030-fead13f69633","Type":"ContainerStarted","Data":"837f1c026eca73ba47b34b562059f8cf4be31efda6d31c65d3d1a15e45cf7127"}
Jul 21 12:41:19.626218 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:19.625805    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-j744v" event={"ID":"eaaabf68-0d82-4671-a030-fead13f69633","Type":"ContainerStarted","Data":"c87a226aedfbfee11a411017f125b49558a36364fc14bc106d312fa28f783950"}
Jul 21 12:41:19.627291 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:19.627268    2556 generic.go:356] "Generic (PLEG): container finished" podID="7438cb69-67ff-4740-aa8f-8501300d0604" containerID="8ca1f878e16d83271c178121ef6e11c80c9007717e54191637babe6e611e9ca3" exitCode=0
Jul 21 12:41:19.627398 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:19.627355    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7438cb69-67ff-4740-aa8f-8501300d0604","Type":"ContainerDied","Data":"8ca1f878e16d83271c178121ef6e11c80c9007717e54191637babe6e611e9ca3"}
Jul 21 12:41:19.646487 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:19.646448    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-56c5594674-j744v" podStartSLOduration=2.347615433 podStartE2EDuration="4.646432726s" totalImagesPullingTime="2.298817293s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:41:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:41:19.645514553 +0000 UTC m=+92.949519695" watchObservedRunningTime="2026-07-21 12:41:19.646432726 +0000 UTC m=+92.950437816"
Jul 21 12:41:20.047801 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:20.047772    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-wx22h"]
Jul 21 12:41:20.050975 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:20.050954    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-wx22h"
Jul 21 12:41:20.053080 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:20.053060    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Jul 21 12:41:20.053080 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:20.053075    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-5k5rz\""
Jul 21 12:41:20.060583 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:20.060564    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-wx22h"]
Jul 21 12:41:20.217137 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:20.217112    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b8bd4d42-5ed6-40eb-b934-83d26a1ce0ca-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-wx22h\" (UID: \"b8bd4d42-5ed6-40eb-b934-83d26a1ce0ca\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-wx22h"
Jul 21 12:41:20.217225 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:20.217158    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jx6rg\" (UniqueName: \"kubernetes.io/projected/b8bd4d42-5ed6-40eb-b934-83d26a1ce0ca-kube-api-access-jx6rg\") pod \"monitoring-plugin-7b7b7bc8bf-wx22h\" (UID: \"b8bd4d42-5ed6-40eb-b934-83d26a1ce0ca\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-wx22h"
Jul 21 12:41:20.318353 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:20.318269    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b8bd4d42-5ed6-40eb-b934-83d26a1ce0ca-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-wx22h\" (UID: \"b8bd4d42-5ed6-40eb-b934-83d26a1ce0ca\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-wx22h"
Jul 21 12:41:20.318353 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:20.318332    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jx6rg\" (UniqueName: \"kubernetes.io/projected/b8bd4d42-5ed6-40eb-b934-83d26a1ce0ca-kube-api-access-jx6rg\") pod \"monitoring-plugin-7b7b7bc8bf-wx22h\" (UID: \"b8bd4d42-5ed6-40eb-b934-83d26a1ce0ca\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-wx22h"
Jul 21 12:41:20.321136 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:20.321111    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b8bd4d42-5ed6-40eb-b934-83d26a1ce0ca-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-wx22h\" (UID: \"b8bd4d42-5ed6-40eb-b934-83d26a1ce0ca\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-wx22h"
Jul 21 12:41:20.327886 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:20.327865    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jx6rg\" (UniqueName: \"kubernetes.io/projected/b8bd4d42-5ed6-40eb-b934-83d26a1ce0ca-kube-api-access-jx6rg\") pod \"monitoring-plugin-7b7b7bc8bf-wx22h\" (UID: \"b8bd4d42-5ed6-40eb-b934-83d26a1ce0ca\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-wx22h"
Jul 21 12:41:20.360659 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:20.360642    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-wx22h"
Jul 21 12:41:20.510880 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:20.510856    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-wx22h"]
Jul 21 12:41:20.513322 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:41:20.513300    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8bd4d42_5ed6_40eb_b934_83d26a1ce0ca.slice/crio-1e90ed0abc997929c777aebdf8a7eefd08944f78fbb82d49b3d7f1dffcfc82ea WatchSource:0}: Error finding container 1e90ed0abc997929c777aebdf8a7eefd08944f78fbb82d49b3d7f1dffcfc82ea: Status 404 returned error can't find the container with id 1e90ed0abc997929c777aebdf8a7eefd08944f78fbb82d49b3d7f1dffcfc82ea
Jul 21 12:41:20.634729 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:20.634636    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-wx22h" event={"ID":"b8bd4d42-5ed6-40eb-b934-83d26a1ce0ca","Type":"ContainerStarted","Data":"1e90ed0abc997929c777aebdf8a7eefd08944f78fbb82d49b3d7f1dffcfc82ea"}
Jul 21 12:41:21.644355 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:21.644314    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7438cb69-67ff-4740-aa8f-8501300d0604","Type":"ContainerStarted","Data":"0b094d3b02188848898853cd1f959f28298083c0c62e6e6d09802066e40e4df7"}
Jul 21 12:41:21.644733 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:21.644391    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7438cb69-67ff-4740-aa8f-8501300d0604","Type":"ContainerStarted","Data":"6945de6229dae2755452aa1ea0f1fd48de0e3fefd13a8dd03f5ec76ca95b70c3"}
Jul 21 12:41:21.644733 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:21.644420    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7438cb69-67ff-4740-aa8f-8501300d0604","Type":"ContainerStarted","Data":"7df67234f806c7f9d14371da095796fba5d9c8abe7915fc72455a299c3c12436"}
Jul 21 12:41:21.644733 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:21.644455    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7438cb69-67ff-4740-aa8f-8501300d0604","Type":"ContainerStarted","Data":"6041567d1c68a5f79f6e1738f32066806fe7e1acdf8dcac55a636eb8238dbded"}
Jul 21 12:41:21.644733 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:21.644489    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7438cb69-67ff-4740-aa8f-8501300d0604","Type":"ContainerStarted","Data":"ba1c2618ebb02a62cc165851902495df1df5faca228120a2ecd9233e3a355e27"}
Jul 21 12:41:22.542207 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:22.542185    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-f4c5cdd74-l4dbz"
Jul 21 12:41:22.651555 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:22.651470    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7438cb69-67ff-4740-aa8f-8501300d0604","Type":"ContainerStarted","Data":"8b3528f3dbe46efe95a7b12cafbef9b44b00538617c6eb443e2fbd133c2002a3"}
Jul 21 12:41:22.653027 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:22.652990    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-wx22h" event={"ID":"b8bd4d42-5ed6-40eb-b934-83d26a1ce0ca","Type":"ContainerStarted","Data":"ea9f1f4465949d7cf3f71fdb35e8cff7ce7d13f278e6e80622be4872af6bcd35"}
Jul 21 12:41:22.653201 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:22.653183    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-wx22h"
Jul 21 12:41:22.658375 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:22.658357    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-wx22h"
Jul 21 12:41:22.685734 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:22.685687    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.467604525 podStartE2EDuration="6.685671466s" totalImagesPullingTime="4.195719059s" totalInitContainerRuntime="22.347882ms" isStatefulPod=false podCreationTimestamp="2026-07-21 12:41:16 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:41:22.684173258 +0000 UTC m=+95.988178366" watchObservedRunningTime="2026-07-21 12:41:22.685671466 +0000 UTC m=+95.989676557"
Jul 21 12:41:22.705601 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:22.705550    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-wx22h" podStartSLOduration=0.871177422 podStartE2EDuration="2.705530879s" totalImagesPullingTime="1.834353457s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:41:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:41:22.704231527 +0000 UTC m=+96.008236640" watchObservedRunningTime="2026-07-21 12:41:22.705530879 +0000 UTC m=+96.009535971"
Jul 21 12:41:24.465772 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:24.465733    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-5cffd5944d-w72zg" podUID="352a97a9-8685-40eb-a22e-f0ed802f3cae" containerName="registry" containerID="cri-o://e89dcf0c8419a2c4e90db56c76b9a48ad05b04c64bb2a4b93ae6282077d8a44d" gracePeriod=30
Jul 21 12:41:24.664400 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:24.664372    2556 generic.go:356] "Generic (PLEG): container finished" podID="352a97a9-8685-40eb-a22e-f0ed802f3cae" containerID="e89dcf0c8419a2c4e90db56c76b9a48ad05b04c64bb2a4b93ae6282077d8a44d" exitCode=0
Jul 21 12:41:24.664526 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:24.664432    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5cffd5944d-w72zg" event={"ID":"352a97a9-8685-40eb-a22e-f0ed802f3cae","Type":"ContainerDied","Data":"e89dcf0c8419a2c4e90db56c76b9a48ad05b04c64bb2a4b93ae6282077d8a44d"}
Jul 21 12:41:24.709634 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:24.709614    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5cffd5944d-w72zg"
Jul 21 12:41:24.751863 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:24.751844    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/352a97a9-8685-40eb-a22e-f0ed802f3cae-installation-pull-secrets\") pod \"352a97a9-8685-40eb-a22e-f0ed802f3cae\" (UID: \"352a97a9-8685-40eb-a22e-f0ed802f3cae\") "
Jul 21 12:41:24.751989 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:24.751894    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/352a97a9-8685-40eb-a22e-f0ed802f3cae-trusted-ca\") pod \"352a97a9-8685-40eb-a22e-f0ed802f3cae\" (UID: \"352a97a9-8685-40eb-a22e-f0ed802f3cae\") "
Jul 21 12:41:24.752076 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:24.752030    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/352a97a9-8685-40eb-a22e-f0ed802f3cae-registry-certificates\") pod \"352a97a9-8685-40eb-a22e-f0ed802f3cae\" (UID: \"352a97a9-8685-40eb-a22e-f0ed802f3cae\") "
Jul 21 12:41:24.752076 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:24.752057    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wqfll\" (UniqueName: \"kubernetes.io/projected/352a97a9-8685-40eb-a22e-f0ed802f3cae-kube-api-access-wqfll\") pod \"352a97a9-8685-40eb-a22e-f0ed802f3cae\" (UID: \"352a97a9-8685-40eb-a22e-f0ed802f3cae\") "
Jul 21 12:41:24.752187 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:24.752080    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/352a97a9-8685-40eb-a22e-f0ed802f3cae-bound-sa-token\") pod \"352a97a9-8685-40eb-a22e-f0ed802f3cae\" (UID: \"352a97a9-8685-40eb-a22e-f0ed802f3cae\") "
Jul 21 12:41:24.752187 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:24.752101    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/352a97a9-8685-40eb-a22e-f0ed802f3cae-image-registry-private-configuration\") pod \"352a97a9-8685-40eb-a22e-f0ed802f3cae\" (UID: \"352a97a9-8685-40eb-a22e-f0ed802f3cae\") "
Jul 21 12:41:24.752187 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:24.752128    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/352a97a9-8685-40eb-a22e-f0ed802f3cae-ca-trust-extracted\") pod \"352a97a9-8685-40eb-a22e-f0ed802f3cae\" (UID: \"352a97a9-8685-40eb-a22e-f0ed802f3cae\") "
Jul 21 12:41:24.752187 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:24.752181    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/352a97a9-8685-40eb-a22e-f0ed802f3cae-registry-tls\") pod \"352a97a9-8685-40eb-a22e-f0ed802f3cae\" (UID: \"352a97a9-8685-40eb-a22e-f0ed802f3cae\") "
Jul 21 12:41:24.752412 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:24.752391    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/352a97a9-8685-40eb-a22e-f0ed802f3cae-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "352a97a9-8685-40eb-a22e-f0ed802f3cae" (UID: "352a97a9-8685-40eb-a22e-f0ed802f3cae"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 12:41:24.752744 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:24.752531    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/352a97a9-8685-40eb-a22e-f0ed802f3cae-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "352a97a9-8685-40eb-a22e-f0ed802f3cae" (UID: "352a97a9-8685-40eb-a22e-f0ed802f3cae"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 12:41:24.754725 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:24.754698    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/352a97a9-8685-40eb-a22e-f0ed802f3cae-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "352a97a9-8685-40eb-a22e-f0ed802f3cae" (UID: "352a97a9-8685-40eb-a22e-f0ed802f3cae"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 12:41:24.754987 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:24.754957    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/352a97a9-8685-40eb-a22e-f0ed802f3cae-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "352a97a9-8685-40eb-a22e-f0ed802f3cae" (UID: "352a97a9-8685-40eb-a22e-f0ed802f3cae"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 12:41:24.755096 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:24.755030    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/352a97a9-8685-40eb-a22e-f0ed802f3cae-kube-api-access-wqfll" (OuterVolumeSpecName: "kube-api-access-wqfll") pod "352a97a9-8685-40eb-a22e-f0ed802f3cae" (UID: "352a97a9-8685-40eb-a22e-f0ed802f3cae"). InnerVolumeSpecName "kube-api-access-wqfll". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 12:41:24.755147 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:24.755100    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/352a97a9-8685-40eb-a22e-f0ed802f3cae-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "352a97a9-8685-40eb-a22e-f0ed802f3cae" (UID: "352a97a9-8685-40eb-a22e-f0ed802f3cae"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 12:41:24.755252 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:24.755236    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/352a97a9-8685-40eb-a22e-f0ed802f3cae-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "352a97a9-8685-40eb-a22e-f0ed802f3cae" (UID: "352a97a9-8685-40eb-a22e-f0ed802f3cae"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 12:41:24.760737 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:24.760717    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/352a97a9-8685-40eb-a22e-f0ed802f3cae-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "352a97a9-8685-40eb-a22e-f0ed802f3cae" (UID: "352a97a9-8685-40eb-a22e-f0ed802f3cae"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 12:41:24.853691 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:24.853669    2556 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/352a97a9-8685-40eb-a22e-f0ed802f3cae-installation-pull-secrets\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Jul 21 12:41:24.853691 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:24.853688    2556 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/352a97a9-8685-40eb-a22e-f0ed802f3cae-trusted-ca\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Jul 21 12:41:24.853800 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:24.853698    2556 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/352a97a9-8685-40eb-a22e-f0ed802f3cae-registry-certificates\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Jul 21 12:41:24.853800 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:24.853706    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wqfll\" (UniqueName: \"kubernetes.io/projected/352a97a9-8685-40eb-a22e-f0ed802f3cae-kube-api-access-wqfll\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Jul 21 12:41:24.853800 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:24.853714    2556 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/352a97a9-8685-40eb-a22e-f0ed802f3cae-bound-sa-token\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Jul 21 12:41:24.853800 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:24.853724    2556 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/352a97a9-8685-40eb-a22e-f0ed802f3cae-image-registry-private-configuration\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Jul 21 12:41:24.853800 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:24.853733    2556 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/352a97a9-8685-40eb-a22e-f0ed802f3cae-ca-trust-extracted\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Jul 21 12:41:24.853800 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:24.853740    2556 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/352a97a9-8685-40eb-a22e-f0ed802f3cae-registry-tls\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Jul 21 12:41:25.670364 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:25.670330    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5cffd5944d-w72zg" event={"ID":"352a97a9-8685-40eb-a22e-f0ed802f3cae","Type":"ContainerDied","Data":"dbb2111af8944e3aec67351e8d99a4bdcf64e9a17ab3bd7fc1d2ea4c8ba01735"}
Jul 21 12:41:25.670836 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:25.670390    2556 scope.go:117] "RemoveContainer" containerID="e89dcf0c8419a2c4e90db56c76b9a48ad05b04c64bb2a4b93ae6282077d8a44d"
Jul 21 12:41:25.670836 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:25.670391    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5cffd5944d-w72zg"
Jul 21 12:41:25.702731 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:25.702700    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5cffd5944d-w72zg"]
Jul 21 12:41:25.717099 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:25.717073    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-5cffd5944d-w72zg"]
Jul 21 12:41:27.193075 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:27.192948    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="352a97a9-8685-40eb-a22e-f0ed802f3cae" path="/var/lib/kubelet/pods/352a97a9-8685-40eb-a22e-f0ed802f3cae/volumes"
Jul 21 12:41:28.806612 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:28.806580    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-695cb7656f-w7rvf"]
Jul 21 12:41:28.806998 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:28.806884    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="352a97a9-8685-40eb-a22e-f0ed802f3cae" containerName="registry"
Jul 21 12:41:28.806998 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:28.806897    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="352a97a9-8685-40eb-a22e-f0ed802f3cae" containerName="registry"
Jul 21 12:41:28.806998 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:28.806956    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="352a97a9-8685-40eb-a22e-f0ed802f3cae" containerName="registry"
Jul 21 12:41:28.848317 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:28.848293    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-695cb7656f-w7rvf"]
Jul 21 12:41:28.848441 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:28.848415    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-695cb7656f-w7rvf"
Jul 21 12:41:28.850938 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:28.850918    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 21 12:41:28.851285 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:28.851272    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 21 12:41:28.851556 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:28.851536    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 21 12:41:28.851860 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:28.851831    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 21 12:41:28.851954 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:28.851885    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 21 12:41:28.851954 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:28.851913    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 21 12:41:28.852292 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:28.852272    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 21 12:41:28.852452 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:28.852434    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-9m7pl\""
Jul 21 12:41:28.880654 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:28.880634    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8a82abb6-3736-4955-9a1a-21a2b8c9383c-oauth-serving-cert\") pod \"console-695cb7656f-w7rvf\" (UID: \"8a82abb6-3736-4955-9a1a-21a2b8c9383c\") " pod="openshift-console/console-695cb7656f-w7rvf"
Jul 21 12:41:28.880748 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:28.880677    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8a82abb6-3736-4955-9a1a-21a2b8c9383c-console-config\") pod \"console-695cb7656f-w7rvf\" (UID: \"8a82abb6-3736-4955-9a1a-21a2b8c9383c\") " pod="openshift-console/console-695cb7656f-w7rvf"
Jul 21 12:41:28.880748 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:28.880695    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-97gf6\" (UniqueName: \"kubernetes.io/projected/8a82abb6-3736-4955-9a1a-21a2b8c9383c-kube-api-access-97gf6\") pod \"console-695cb7656f-w7rvf\" (UID: \"8a82abb6-3736-4955-9a1a-21a2b8c9383c\") " pod="openshift-console/console-695cb7656f-w7rvf"
Jul 21 12:41:28.880844 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:28.880750    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8a82abb6-3736-4955-9a1a-21a2b8c9383c-service-ca\") pod \"console-695cb7656f-w7rvf\" (UID: \"8a82abb6-3736-4955-9a1a-21a2b8c9383c\") " pod="openshift-console/console-695cb7656f-w7rvf"
Jul 21 12:41:28.880844 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:28.880820    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8a82abb6-3736-4955-9a1a-21a2b8c9383c-console-serving-cert\") pod \"console-695cb7656f-w7rvf\" (UID: \"8a82abb6-3736-4955-9a1a-21a2b8c9383c\") " pod="openshift-console/console-695cb7656f-w7rvf"
Jul 21 12:41:28.880913 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:28.880843    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8a82abb6-3736-4955-9a1a-21a2b8c9383c-console-oauth-config\") pod \"console-695cb7656f-w7rvf\" (UID: \"8a82abb6-3736-4955-9a1a-21a2b8c9383c\") " pod="openshift-console/console-695cb7656f-w7rvf"
Jul 21 12:41:28.981666 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:28.981634    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8a82abb6-3736-4955-9a1a-21a2b8c9383c-oauth-serving-cert\") pod \"console-695cb7656f-w7rvf\" (UID: \"8a82abb6-3736-4955-9a1a-21a2b8c9383c\") " pod="openshift-console/console-695cb7656f-w7rvf"
Jul 21 12:41:28.981762 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:28.981687    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8a82abb6-3736-4955-9a1a-21a2b8c9383c-console-config\") pod \"console-695cb7656f-w7rvf\" (UID: \"8a82abb6-3736-4955-9a1a-21a2b8c9383c\") " pod="openshift-console/console-695cb7656f-w7rvf"
Jul 21 12:41:28.981762 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:28.981705    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-97gf6\" (UniqueName: \"kubernetes.io/projected/8a82abb6-3736-4955-9a1a-21a2b8c9383c-kube-api-access-97gf6\") pod \"console-695cb7656f-w7rvf\" (UID: \"8a82abb6-3736-4955-9a1a-21a2b8c9383c\") " pod="openshift-console/console-695cb7656f-w7rvf"
Jul 21 12:41:28.981762 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:28.981728    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8a82abb6-3736-4955-9a1a-21a2b8c9383c-service-ca\") pod \"console-695cb7656f-w7rvf\" (UID: \"8a82abb6-3736-4955-9a1a-21a2b8c9383c\") " pod="openshift-console/console-695cb7656f-w7rvf"
Jul 21 12:41:28.981762 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:28.981750    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8a82abb6-3736-4955-9a1a-21a2b8c9383c-console-serving-cert\") pod \"console-695cb7656f-w7rvf\" (UID: \"8a82abb6-3736-4955-9a1a-21a2b8c9383c\") " pod="openshift-console/console-695cb7656f-w7rvf"
Jul 21 12:41:28.981955 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:28.981784    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8a82abb6-3736-4955-9a1a-21a2b8c9383c-console-oauth-config\") pod \"console-695cb7656f-w7rvf\" (UID: \"8a82abb6-3736-4955-9a1a-21a2b8c9383c\") " pod="openshift-console/console-695cb7656f-w7rvf"
Jul 21 12:41:28.982859 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:28.982834    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8a82abb6-3736-4955-9a1a-21a2b8c9383c-service-ca\") pod \"console-695cb7656f-w7rvf\" (UID: \"8a82abb6-3736-4955-9a1a-21a2b8c9383c\") " pod="openshift-console/console-695cb7656f-w7rvf"
Jul 21 12:41:28.983108 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:28.983082    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8a82abb6-3736-4955-9a1a-21a2b8c9383c-oauth-serving-cert\") pod \"console-695cb7656f-w7rvf\" (UID: \"8a82abb6-3736-4955-9a1a-21a2b8c9383c\") " pod="openshift-console/console-695cb7656f-w7rvf"
Jul 21 12:41:28.983171 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:28.983117    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8a82abb6-3736-4955-9a1a-21a2b8c9383c-console-config\") pod \"console-695cb7656f-w7rvf\" (UID: \"8a82abb6-3736-4955-9a1a-21a2b8c9383c\") " pod="openshift-console/console-695cb7656f-w7rvf"
Jul 21 12:41:28.987993 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:28.984728    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8a82abb6-3736-4955-9a1a-21a2b8c9383c-console-oauth-config\") pod \"console-695cb7656f-w7rvf\" (UID: \"8a82abb6-3736-4955-9a1a-21a2b8c9383c\") " pod="openshift-console/console-695cb7656f-w7rvf"
Jul 21 12:41:28.987993 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:28.984868    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8a82abb6-3736-4955-9a1a-21a2b8c9383c-console-serving-cert\") pod \"console-695cb7656f-w7rvf\" (UID: \"8a82abb6-3736-4955-9a1a-21a2b8c9383c\") " pod="openshift-console/console-695cb7656f-w7rvf"
Jul 21 12:41:28.991670 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:28.991645    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-97gf6\" (UniqueName: \"kubernetes.io/projected/8a82abb6-3736-4955-9a1a-21a2b8c9383c-kube-api-access-97gf6\") pod \"console-695cb7656f-w7rvf\" (UID: \"8a82abb6-3736-4955-9a1a-21a2b8c9383c\") " pod="openshift-console/console-695cb7656f-w7rvf"
Jul 21 12:41:29.158559 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:29.158505    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-695cb7656f-w7rvf"
Jul 21 12:41:29.319741 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:29.319712    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-695cb7656f-w7rvf"]
Jul 21 12:41:29.323945 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:41:29.323917    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a82abb6_3736_4955_9a1a_21a2b8c9383c.slice/crio-1cce18e8d2a46c2df6eb7ff98ede4efb128e56a4a01b422a038bd3b7a7c9ef2b WatchSource:0}: Error finding container 1cce18e8d2a46c2df6eb7ff98ede4efb128e56a4a01b422a038bd3b7a7c9ef2b: Status 404 returned error can't find the container with id 1cce18e8d2a46c2df6eb7ff98ede4efb128e56a4a01b422a038bd3b7a7c9ef2b
Jul 21 12:41:29.687737 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:29.687709    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-695cb7656f-w7rvf" event={"ID":"8a82abb6-3736-4955-9a1a-21a2b8c9383c","Type":"ContainerStarted","Data":"1cce18e8d2a46c2df6eb7ff98ede4efb128e56a4a01b422a038bd3b7a7c9ef2b"}
Jul 21 12:41:32.540161 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:32.540134    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-clxmp"
Jul 21 12:41:33.707112 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:33.707063    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-695cb7656f-w7rvf" event={"ID":"8a82abb6-3736-4955-9a1a-21a2b8c9383c","Type":"ContainerStarted","Data":"97b78023ebe2ea8a9f8c417e0755719ddcd2951227d90afcff0d02733582ac78"}
Jul 21 12:41:33.729024 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:33.728968    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-695cb7656f-w7rvf" podStartSLOduration=2.019035808 podStartE2EDuration="5.728951544s" totalImagesPullingTime="3.709915736s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:41:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:41:33.727587207 +0000 UTC m=+107.031592299" watchObservedRunningTime="2026-07-21 12:41:33.728951544 +0000 UTC m=+107.032956634"
Jul 21 12:41:39.158631 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:39.158599    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-695cb7656f-w7rvf"
Jul 21 12:41:39.158985 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:39.158672    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-695cb7656f-w7rvf"
Jul 21 12:41:39.163223 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:39.163202    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-695cb7656f-w7rvf"
Jul 21 12:41:39.734533 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:39.734511    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-695cb7656f-w7rvf"
Jul 21 12:41:45.755915 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:45.755874    2556 generic.go:356] "Generic (PLEG): container finished" podID="277500e3-0956-4360-90a2-6512c5725349" containerID="646cf7365c3c41f4debcf3fab58a7612379d8d61823a46e59e255a01fae54d7c" exitCode=0
Jul 21 12:41:45.756293 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:45.755952    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-rn8m5" event={"ID":"277500e3-0956-4360-90a2-6512c5725349","Type":"ContainerDied","Data":"646cf7365c3c41f4debcf3fab58a7612379d8d61823a46e59e255a01fae54d7c"}
Jul 21 12:41:45.756347 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:45.756333    2556 scope.go:117] "RemoveContainer" containerID="646cf7365c3c41f4debcf3fab58a7612379d8d61823a46e59e255a01fae54d7c"
Jul 21 12:41:46.633454 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:46.633423    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-695cb7656f-w7rvf"]
Jul 21 12:41:46.761452 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:46.761425    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-rn8m5" event={"ID":"277500e3-0956-4360-90a2-6512c5725349","Type":"ContainerStarted","Data":"bc16d25aee764f7aa4f62bf77c492948f509b2aebb96488c31af27fc6c5bec13"}
Jul 21 12:41:50.780814 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:50.780778    2556 generic.go:356] "Generic (PLEG): container finished" podID="8fe9a2da-76ca-4b0c-b86a-4dc3dad01c2e" containerID="7f419aca93b3d3b96be5179a341b4ffce4f3a0597d3e6df39468af1ef79b3382" exitCode=0
Jul 21 12:41:50.781205 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:50.780853    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-s7gwh" event={"ID":"8fe9a2da-76ca-4b0c-b86a-4dc3dad01c2e","Type":"ContainerDied","Data":"7f419aca93b3d3b96be5179a341b4ffce4f3a0597d3e6df39468af1ef79b3382"}
Jul 21 12:41:50.781258 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:50.781205    2556 scope.go:117] "RemoveContainer" containerID="7f419aca93b3d3b96be5179a341b4ffce4f3a0597d3e6df39468af1ef79b3382"
Jul 21 12:41:51.786630 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:41:51.786598    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-s7gwh" event={"ID":"8fe9a2da-76ca-4b0c-b86a-4dc3dad01c2e","Type":"ContainerStarted","Data":"9e69a2439c48988a18226f8e8854a7fead964cf6ea5ca6ec180361dd66f4ffa5"}
Jul 21 12:42:01.827873 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:01.827834    2556 generic.go:356] "Generic (PLEG): container finished" podID="7793bdb2-35aa-476a-a73c-95856526fff3" containerID="60ef621461807bf3b9765b9750a1a9cbdea5304128d66d43eba30ab000cce689" exitCode=0
Jul 21 12:42:01.828425 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:01.827921    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-8zqrp" event={"ID":"7793bdb2-35aa-476a-a73c-95856526fff3","Type":"ContainerDied","Data":"60ef621461807bf3b9765b9750a1a9cbdea5304128d66d43eba30ab000cce689"}
Jul 21 12:42:01.828425 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:01.828388    2556 scope.go:117] "RemoveContainer" containerID="60ef621461807bf3b9765b9750a1a9cbdea5304128d66d43eba30ab000cce689"
Jul 21 12:42:02.834749 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:02.834712    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-8zqrp" event={"ID":"7793bdb2-35aa-476a-a73c-95856526fff3","Type":"ContainerStarted","Data":"2a92af9cfd9328d5514e36893007f9205c177fb9bcb3b90c3fa2d4f3a15526ab"}
Jul 21 12:42:11.654880 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:11.654828    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-695cb7656f-w7rvf" podUID="8a82abb6-3736-4955-9a1a-21a2b8c9383c" containerName="console" containerID="cri-o://97b78023ebe2ea8a9f8c417e0755719ddcd2951227d90afcff0d02733582ac78" gracePeriod=15
Jul 21 12:42:11.870629 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:11.870603    2556 generic.go:356] "Generic (PLEG): container finished" podID="8a82abb6-3736-4955-9a1a-21a2b8c9383c" containerID="97b78023ebe2ea8a9f8c417e0755719ddcd2951227d90afcff0d02733582ac78" exitCode=0
Jul 21 12:42:11.870720 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:11.870647    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-695cb7656f-w7rvf" event={"ID":"8a82abb6-3736-4955-9a1a-21a2b8c9383c","Type":"ContainerDied","Data":"97b78023ebe2ea8a9f8c417e0755719ddcd2951227d90afcff0d02733582ac78"}
Jul 21 12:42:11.887153 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:11.887130    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-695cb7656f-w7rvf"
Jul 21 12:42:12.000417 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:12.000396    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8a82abb6-3736-4955-9a1a-21a2b8c9383c-console-serving-cert\") pod \"8a82abb6-3736-4955-9a1a-21a2b8c9383c\" (UID: \"8a82abb6-3736-4955-9a1a-21a2b8c9383c\") "
Jul 21 12:42:12.000509 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:12.000429    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8a82abb6-3736-4955-9a1a-21a2b8c9383c-console-oauth-config\") pod \"8a82abb6-3736-4955-9a1a-21a2b8c9383c\" (UID: \"8a82abb6-3736-4955-9a1a-21a2b8c9383c\") "
Jul 21 12:42:12.000509 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:12.000454    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8a82abb6-3736-4955-9a1a-21a2b8c9383c-service-ca\") pod \"8a82abb6-3736-4955-9a1a-21a2b8c9383c\" (UID: \"8a82abb6-3736-4955-9a1a-21a2b8c9383c\") "
Jul 21 12:42:12.000578 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:12.000483    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8a82abb6-3736-4955-9a1a-21a2b8c9383c-oauth-serving-cert\") pod \"8a82abb6-3736-4955-9a1a-21a2b8c9383c\" (UID: \"8a82abb6-3736-4955-9a1a-21a2b8c9383c\") "
Jul 21 12:42:12.000578 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:12.000547    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-97gf6\" (UniqueName: \"kubernetes.io/projected/8a82abb6-3736-4955-9a1a-21a2b8c9383c-kube-api-access-97gf6\") pod \"8a82abb6-3736-4955-9a1a-21a2b8c9383c\" (UID: \"8a82abb6-3736-4955-9a1a-21a2b8c9383c\") "
Jul 21 12:42:12.000654 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:12.000636    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8a82abb6-3736-4955-9a1a-21a2b8c9383c-console-config\") pod \"8a82abb6-3736-4955-9a1a-21a2b8c9383c\" (UID: \"8a82abb6-3736-4955-9a1a-21a2b8c9383c\") "
Jul 21 12:42:12.000858 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:12.000811    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8a82abb6-3736-4955-9a1a-21a2b8c9383c-service-ca" (OuterVolumeSpecName: "service-ca") pod "8a82abb6-3736-4955-9a1a-21a2b8c9383c" (UID: "8a82abb6-3736-4955-9a1a-21a2b8c9383c"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 12:42:12.001101 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:12.000872    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8a82abb6-3736-4955-9a1a-21a2b8c9383c-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "8a82abb6-3736-4955-9a1a-21a2b8c9383c" (UID: "8a82abb6-3736-4955-9a1a-21a2b8c9383c"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 12:42:12.001164 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:12.001093    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8a82abb6-3736-4955-9a1a-21a2b8c9383c-console-config" (OuterVolumeSpecName: "console-config") pod "8a82abb6-3736-4955-9a1a-21a2b8c9383c" (UID: "8a82abb6-3736-4955-9a1a-21a2b8c9383c"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 12:42:12.002859 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:12.002835    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8a82abb6-3736-4955-9a1a-21a2b8c9383c-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "8a82abb6-3736-4955-9a1a-21a2b8c9383c" (UID: "8a82abb6-3736-4955-9a1a-21a2b8c9383c"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 12:42:12.002927 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:12.002848    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8a82abb6-3736-4955-9a1a-21a2b8c9383c-kube-api-access-97gf6" (OuterVolumeSpecName: "kube-api-access-97gf6") pod "8a82abb6-3736-4955-9a1a-21a2b8c9383c" (UID: "8a82abb6-3736-4955-9a1a-21a2b8c9383c"). InnerVolumeSpecName "kube-api-access-97gf6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 12:42:12.002927 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:12.002861    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8a82abb6-3736-4955-9a1a-21a2b8c9383c-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "8a82abb6-3736-4955-9a1a-21a2b8c9383c" (UID: "8a82abb6-3736-4955-9a1a-21a2b8c9383c"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 12:42:12.101645 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:12.101623    2556 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8a82abb6-3736-4955-9a1a-21a2b8c9383c-console-oauth-config\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Jul 21 12:42:12.101645 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:12.101644    2556 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8a82abb6-3736-4955-9a1a-21a2b8c9383c-service-ca\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Jul 21 12:42:12.101767 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:12.101653    2556 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8a82abb6-3736-4955-9a1a-21a2b8c9383c-oauth-serving-cert\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Jul 21 12:42:12.101767 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:12.101662    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-97gf6\" (UniqueName: \"kubernetes.io/projected/8a82abb6-3736-4955-9a1a-21a2b8c9383c-kube-api-access-97gf6\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Jul 21 12:42:12.101767 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:12.101671    2556 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8a82abb6-3736-4955-9a1a-21a2b8c9383c-console-config\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Jul 21 12:42:12.101767 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:12.101679    2556 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8a82abb6-3736-4955-9a1a-21a2b8c9383c-console-serving-cert\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Jul 21 12:42:12.876150 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:12.876107    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-695cb7656f-w7rvf" event={"ID":"8a82abb6-3736-4955-9a1a-21a2b8c9383c","Type":"ContainerDied","Data":"1cce18e8d2a46c2df6eb7ff98ede4efb128e56a4a01b422a038bd3b7a7c9ef2b"}
Jul 21 12:42:12.876602 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:12.876174    2556 scope.go:117] "RemoveContainer" containerID="97b78023ebe2ea8a9f8c417e0755719ddcd2951227d90afcff0d02733582ac78"
Jul 21 12:42:12.876602 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:12.876127    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-695cb7656f-w7rvf"
Jul 21 12:42:12.899175 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:12.899146    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-695cb7656f-w7rvf"]
Jul 21 12:42:12.902046 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:12.901996    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-695cb7656f-w7rvf"]
Jul 21 12:42:13.193879 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:13.193814    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8a82abb6-3736-4955-9a1a-21a2b8c9383c" path="/var/lib/kubelet/pods/8a82abb6-3736-4955-9a1a-21a2b8c9383c/volumes"
Jul 21 12:42:35.780520 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:35.780485    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 21 12:42:35.781068 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:35.780846    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="7438cb69-67ff-4740-aa8f-8501300d0604" containerName="alertmanager" containerID="cri-o://ba1c2618ebb02a62cc165851902495df1df5faca228120a2ecd9233e3a355e27" gracePeriod=120
Jul 21 12:42:35.781068 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:35.780898    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="7438cb69-67ff-4740-aa8f-8501300d0604" containerName="kube-rbac-proxy-metric" containerID="cri-o://0b094d3b02188848898853cd1f959f28298083c0c62e6e6d09802066e40e4df7" gracePeriod=120
Jul 21 12:42:35.781068 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:35.780920    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="7438cb69-67ff-4740-aa8f-8501300d0604" containerName="prom-label-proxy" containerID="cri-o://8b3528f3dbe46efe95a7b12cafbef9b44b00538617c6eb443e2fbd133c2002a3" gracePeriod=120
Jul 21 12:42:35.781068 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:35.780934    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="7438cb69-67ff-4740-aa8f-8501300d0604" containerName="kube-rbac-proxy-web" containerID="cri-o://7df67234f806c7f9d14371da095796fba5d9c8abe7915fc72455a299c3c12436" gracePeriod=120
Jul 21 12:42:35.781068 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:35.780955    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="7438cb69-67ff-4740-aa8f-8501300d0604" containerName="kube-rbac-proxy" containerID="cri-o://6945de6229dae2755452aa1ea0f1fd48de0e3fefd13a8dd03f5ec76ca95b70c3" gracePeriod=120
Jul 21 12:42:35.781368 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:35.780968    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="7438cb69-67ff-4740-aa8f-8501300d0604" containerName="config-reloader" containerID="cri-o://6041567d1c68a5f79f6e1738f32066806fe7e1acdf8dcac55a636eb8238dbded" gracePeriod=120
Jul 21 12:42:35.961385 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:35.961359    2556 generic.go:356] "Generic (PLEG): container finished" podID="7438cb69-67ff-4740-aa8f-8501300d0604" containerID="8b3528f3dbe46efe95a7b12cafbef9b44b00538617c6eb443e2fbd133c2002a3" exitCode=0
Jul 21 12:42:35.961385 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:35.961380    2556 generic.go:356] "Generic (PLEG): container finished" podID="7438cb69-67ff-4740-aa8f-8501300d0604" containerID="6945de6229dae2755452aa1ea0f1fd48de0e3fefd13a8dd03f5ec76ca95b70c3" exitCode=0
Jul 21 12:42:35.961385 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:35.961385    2556 generic.go:356] "Generic (PLEG): container finished" podID="7438cb69-67ff-4740-aa8f-8501300d0604" containerID="6041567d1c68a5f79f6e1738f32066806fe7e1acdf8dcac55a636eb8238dbded" exitCode=0
Jul 21 12:42:35.961552 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:35.961391    2556 generic.go:356] "Generic (PLEG): container finished" podID="7438cb69-67ff-4740-aa8f-8501300d0604" containerID="ba1c2618ebb02a62cc165851902495df1df5faca228120a2ecd9233e3a355e27" exitCode=0
Jul 21 12:42:35.961552 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:35.961434    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7438cb69-67ff-4740-aa8f-8501300d0604","Type":"ContainerDied","Data":"8b3528f3dbe46efe95a7b12cafbef9b44b00538617c6eb443e2fbd133c2002a3"}
Jul 21 12:42:35.961552 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:35.961488    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7438cb69-67ff-4740-aa8f-8501300d0604","Type":"ContainerDied","Data":"6945de6229dae2755452aa1ea0f1fd48de0e3fefd13a8dd03f5ec76ca95b70c3"}
Jul 21 12:42:35.961552 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:35.961513    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7438cb69-67ff-4740-aa8f-8501300d0604","Type":"ContainerDied","Data":"6041567d1c68a5f79f6e1738f32066806fe7e1acdf8dcac55a636eb8238dbded"}
Jul 21 12:42:35.961552 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:35.961539    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7438cb69-67ff-4740-aa8f-8501300d0604","Type":"ContainerDied","Data":"ba1c2618ebb02a62cc165851902495df1df5faca228120a2ecd9233e3a355e27"}
Jul 21 12:42:36.970669 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:36.970640    2556 generic.go:356] "Generic (PLEG): container finished" podID="7438cb69-67ff-4740-aa8f-8501300d0604" containerID="0b094d3b02188848898853cd1f959f28298083c0c62e6e6d09802066e40e4df7" exitCode=0
Jul 21 12:42:36.970669 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:36.970666    2556 generic.go:356] "Generic (PLEG): container finished" podID="7438cb69-67ff-4740-aa8f-8501300d0604" containerID="7df67234f806c7f9d14371da095796fba5d9c8abe7915fc72455a299c3c12436" exitCode=0
Jul 21 12:42:36.971180 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:36.970707    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7438cb69-67ff-4740-aa8f-8501300d0604","Type":"ContainerDied","Data":"0b094d3b02188848898853cd1f959f28298083c0c62e6e6d09802066e40e4df7"}
Jul 21 12:42:36.971180 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:36.970759    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7438cb69-67ff-4740-aa8f-8501300d0604","Type":"ContainerDied","Data":"7df67234f806c7f9d14371da095796fba5d9c8abe7915fc72455a299c3c12436"}
Jul 21 12:42:37.015516 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:37.015498    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:42:37.166986 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:37.166912    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7438cb69-67ff-4740-aa8f-8501300d0604-secret-alertmanager-kube-rbac-proxy-web\") pod \"7438cb69-67ff-4740-aa8f-8501300d0604\" (UID: \"7438cb69-67ff-4740-aa8f-8501300d0604\") "
Jul 21 12:42:37.166986 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:37.166949    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/7438cb69-67ff-4740-aa8f-8501300d0604-cluster-tls-config\") pod \"7438cb69-67ff-4740-aa8f-8501300d0604\" (UID: \"7438cb69-67ff-4740-aa8f-8501300d0604\") "
Jul 21 12:42:37.166986 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:37.166970    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7438cb69-67ff-4740-aa8f-8501300d0604-alertmanager-trusted-ca-bundle\") pod \"7438cb69-67ff-4740-aa8f-8501300d0604\" (UID: \"7438cb69-67ff-4740-aa8f-8501300d0604\") "
Jul 21 12:42:37.167258 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:37.166995    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-872jt\" (UniqueName: \"kubernetes.io/projected/7438cb69-67ff-4740-aa8f-8501300d0604-kube-api-access-872jt\") pod \"7438cb69-67ff-4740-aa8f-8501300d0604\" (UID: \"7438cb69-67ff-4740-aa8f-8501300d0604\") "
Jul 21 12:42:37.167258 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:37.167040    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7438cb69-67ff-4740-aa8f-8501300d0604-web-config\") pod \"7438cb69-67ff-4740-aa8f-8501300d0604\" (UID: \"7438cb69-67ff-4740-aa8f-8501300d0604\") "
Jul 21 12:42:37.167258 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:37.167080    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7438cb69-67ff-4740-aa8f-8501300d0604-secret-alertmanager-kube-rbac-proxy\") pod \"7438cb69-67ff-4740-aa8f-8501300d0604\" (UID: \"7438cb69-67ff-4740-aa8f-8501300d0604\") "
Jul 21 12:42:37.167258 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:37.167108    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7438cb69-67ff-4740-aa8f-8501300d0604-metrics-client-ca\") pod \"7438cb69-67ff-4740-aa8f-8501300d0604\" (UID: \"7438cb69-67ff-4740-aa8f-8501300d0604\") "
Jul 21 12:42:37.167258 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:37.167130    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/7438cb69-67ff-4740-aa8f-8501300d0604-secret-alertmanager-main-tls\") pod \"7438cb69-67ff-4740-aa8f-8501300d0604\" (UID: \"7438cb69-67ff-4740-aa8f-8501300d0604\") "
Jul 21 12:42:37.167258 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:37.167172    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/7438cb69-67ff-4740-aa8f-8501300d0604-config-volume\") pod \"7438cb69-67ff-4740-aa8f-8501300d0604\" (UID: \"7438cb69-67ff-4740-aa8f-8501300d0604\") "
Jul 21 12:42:37.167258 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:37.167200    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/7438cb69-67ff-4740-aa8f-8501300d0604-secret-alertmanager-kube-rbac-proxy-metric\") pod \"7438cb69-67ff-4740-aa8f-8501300d0604\" (UID: \"7438cb69-67ff-4740-aa8f-8501300d0604\") "
Jul 21 12:42:37.167258 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:37.167254    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/7438cb69-67ff-4740-aa8f-8501300d0604-alertmanager-main-db\") pod \"7438cb69-67ff-4740-aa8f-8501300d0604\" (UID: \"7438cb69-67ff-4740-aa8f-8501300d0604\") "
Jul 21 12:42:37.167627 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:37.167301    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7438cb69-67ff-4740-aa8f-8501300d0604-tls-assets\") pod \"7438cb69-67ff-4740-aa8f-8501300d0604\" (UID: \"7438cb69-67ff-4740-aa8f-8501300d0604\") "
Jul 21 12:42:37.167627 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:37.167339    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7438cb69-67ff-4740-aa8f-8501300d0604-config-out\") pod \"7438cb69-67ff-4740-aa8f-8501300d0604\" (UID: \"7438cb69-67ff-4740-aa8f-8501300d0604\") "
Jul 21 12:42:37.167627 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:37.167491    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7438cb69-67ff-4740-aa8f-8501300d0604-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "7438cb69-67ff-4740-aa8f-8501300d0604" (UID: "7438cb69-67ff-4740-aa8f-8501300d0604"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 12:42:37.167627 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:37.167563    2556 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7438cb69-67ff-4740-aa8f-8501300d0604-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Jul 21 12:42:37.167627 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:37.167587    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7438cb69-67ff-4740-aa8f-8501300d0604-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "7438cb69-67ff-4740-aa8f-8501300d0604" (UID: "7438cb69-67ff-4740-aa8f-8501300d0604"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 12:42:37.167881 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:37.167848    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7438cb69-67ff-4740-aa8f-8501300d0604-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "7438cb69-67ff-4740-aa8f-8501300d0604" (UID: "7438cb69-67ff-4740-aa8f-8501300d0604"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 12:42:37.170618 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:37.170553    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7438cb69-67ff-4740-aa8f-8501300d0604-kube-api-access-872jt" (OuterVolumeSpecName: "kube-api-access-872jt") pod "7438cb69-67ff-4740-aa8f-8501300d0604" (UID: "7438cb69-67ff-4740-aa8f-8501300d0604"). InnerVolumeSpecName "kube-api-access-872jt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 12:42:37.172302 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:37.172273    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7438cb69-67ff-4740-aa8f-8501300d0604-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "7438cb69-67ff-4740-aa8f-8501300d0604" (UID: "7438cb69-67ff-4740-aa8f-8501300d0604"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 12:42:37.172409 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:37.172315    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7438cb69-67ff-4740-aa8f-8501300d0604-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "7438cb69-67ff-4740-aa8f-8501300d0604" (UID: "7438cb69-67ff-4740-aa8f-8501300d0604"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 12:42:37.172567 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:37.172543    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7438cb69-67ff-4740-aa8f-8501300d0604-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "7438cb69-67ff-4740-aa8f-8501300d0604" (UID: "7438cb69-67ff-4740-aa8f-8501300d0604"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 12:42:37.172685 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:37.172582    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7438cb69-67ff-4740-aa8f-8501300d0604-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "7438cb69-67ff-4740-aa8f-8501300d0604" (UID: "7438cb69-67ff-4740-aa8f-8501300d0604"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 12:42:37.173218 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:37.173194    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7438cb69-67ff-4740-aa8f-8501300d0604-config-out" (OuterVolumeSpecName: "config-out") pod "7438cb69-67ff-4740-aa8f-8501300d0604" (UID: "7438cb69-67ff-4740-aa8f-8501300d0604"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 12:42:37.173335 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:37.173309    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7438cb69-67ff-4740-aa8f-8501300d0604-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "7438cb69-67ff-4740-aa8f-8501300d0604" (UID: "7438cb69-67ff-4740-aa8f-8501300d0604"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 12:42:37.173464 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:37.173401    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7438cb69-67ff-4740-aa8f-8501300d0604-config-volume" (OuterVolumeSpecName: "config-volume") pod "7438cb69-67ff-4740-aa8f-8501300d0604" (UID: "7438cb69-67ff-4740-aa8f-8501300d0604"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 12:42:37.175548 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:37.175524    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7438cb69-67ff-4740-aa8f-8501300d0604-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "7438cb69-67ff-4740-aa8f-8501300d0604" (UID: "7438cb69-67ff-4740-aa8f-8501300d0604"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 12:42:37.185025 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:37.184996    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7438cb69-67ff-4740-aa8f-8501300d0604-web-config" (OuterVolumeSpecName: "web-config") pod "7438cb69-67ff-4740-aa8f-8501300d0604" (UID: "7438cb69-67ff-4740-aa8f-8501300d0604"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 12:42:37.268066 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:37.268047    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-872jt\" (UniqueName: \"kubernetes.io/projected/7438cb69-67ff-4740-aa8f-8501300d0604-kube-api-access-872jt\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Jul 21 12:42:37.268066 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:37.268065    2556 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7438cb69-67ff-4740-aa8f-8501300d0604-web-config\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Jul 21 12:42:37.268174 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:37.268074    2556 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7438cb69-67ff-4740-aa8f-8501300d0604-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Jul 21 12:42:37.268174 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:37.268083    2556 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7438cb69-67ff-4740-aa8f-8501300d0604-metrics-client-ca\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Jul 21 12:42:37.268174 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:37.268091    2556 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/7438cb69-67ff-4740-aa8f-8501300d0604-secret-alertmanager-main-tls\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Jul 21 12:42:37.268174 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:37.268099    2556 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/7438cb69-67ff-4740-aa8f-8501300d0604-config-volume\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Jul 21 12:42:37.268174 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:37.268106    2556 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/7438cb69-67ff-4740-aa8f-8501300d0604-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Jul 21 12:42:37.268174 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:37.268114    2556 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/7438cb69-67ff-4740-aa8f-8501300d0604-alertmanager-main-db\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Jul 21 12:42:37.268174 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:37.268122    2556 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7438cb69-67ff-4740-aa8f-8501300d0604-tls-assets\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Jul 21 12:42:37.268174 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:37.268129    2556 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7438cb69-67ff-4740-aa8f-8501300d0604-config-out\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Jul 21 12:42:37.268174 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:37.268136    2556 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7438cb69-67ff-4740-aa8f-8501300d0604-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Jul 21 12:42:37.268174 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:37.268144    2556 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/7438cb69-67ff-4740-aa8f-8501300d0604-cluster-tls-config\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Jul 21 12:42:37.977317 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:37.977290    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7438cb69-67ff-4740-aa8f-8501300d0604","Type":"ContainerDied","Data":"eec9a7200f3e236744492a1b3f93757c3f5915be3feb0ff8903a8716ce3d539a"}
Jul 21 12:42:37.977577 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:37.977350    2556 scope.go:117] "RemoveContainer" containerID="8b3528f3dbe46efe95a7b12cafbef9b44b00538617c6eb443e2fbd133c2002a3"
Jul 21 12:42:37.977577 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:37.977303    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:42:37.984935 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:37.984917    2556 scope.go:117] "RemoveContainer" containerID="0b094d3b02188848898853cd1f959f28298083c0c62e6e6d09802066e40e4df7"
Jul 21 12:42:37.991612 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:37.991595    2556 scope.go:117] "RemoveContainer" containerID="6945de6229dae2755452aa1ea0f1fd48de0e3fefd13a8dd03f5ec76ca95b70c3"
Jul 21 12:42:37.998167 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:37.998150    2556 scope.go:117] "RemoveContainer" containerID="7df67234f806c7f9d14371da095796fba5d9c8abe7915fc72455a299c3c12436"
Jul 21 12:42:38.000376 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:38.000356    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 21 12:42:38.005432 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:38.005413    2556 scope.go:117] "RemoveContainer" containerID="6041567d1c68a5f79f6e1738f32066806fe7e1acdf8dcac55a636eb8238dbded"
Jul 21 12:42:38.006254 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:38.006234    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 21 12:42:38.012270 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:38.012254    2556 scope.go:117] "RemoveContainer" containerID="ba1c2618ebb02a62cc165851902495df1df5faca228120a2ecd9233e3a355e27"
Jul 21 12:42:38.019059 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:38.019042    2556 scope.go:117] "RemoveContainer" containerID="8ca1f878e16d83271c178121ef6e11c80c9007717e54191637babe6e611e9ca3"
Jul 21 12:42:38.038945 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:38.038926    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 21 12:42:38.039279 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:38.039266    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8a82abb6-3736-4955-9a1a-21a2b8c9383c" containerName="console"
Jul 21 12:42:38.039332 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:38.039281    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="8a82abb6-3736-4955-9a1a-21a2b8c9383c" containerName="console"
Jul 21 12:42:38.039332 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:38.039289    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7438cb69-67ff-4740-aa8f-8501300d0604" containerName="init-config-reloader"
Jul 21 12:42:38.039332 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:38.039296    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="7438cb69-67ff-4740-aa8f-8501300d0604" containerName="init-config-reloader"
Jul 21 12:42:38.039332 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:38.039315    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7438cb69-67ff-4740-aa8f-8501300d0604" containerName="prom-label-proxy"
Jul 21 12:42:38.039332 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:38.039321    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="7438cb69-67ff-4740-aa8f-8501300d0604" containerName="prom-label-proxy"
Jul 21 12:42:38.039332 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:38.039330    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7438cb69-67ff-4740-aa8f-8501300d0604" containerName="alertmanager"
Jul 21 12:42:38.039332 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:38.039334    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="7438cb69-67ff-4740-aa8f-8501300d0604" containerName="alertmanager"
Jul 21 12:42:38.039519 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:38.039339    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7438cb69-67ff-4740-aa8f-8501300d0604" containerName="config-reloader"
Jul 21 12:42:38.039519 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:38.039344    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="7438cb69-67ff-4740-aa8f-8501300d0604" containerName="config-reloader"
Jul 21 12:42:38.039519 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:38.039353    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7438cb69-67ff-4740-aa8f-8501300d0604" containerName="kube-rbac-proxy-metric"
Jul 21 12:42:38.039519 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:38.039357    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="7438cb69-67ff-4740-aa8f-8501300d0604" containerName="kube-rbac-proxy-metric"
Jul 21 12:42:38.039519 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:38.039364    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7438cb69-67ff-4740-aa8f-8501300d0604" containerName="kube-rbac-proxy-web"
Jul 21 12:42:38.039519 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:38.039369    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="7438cb69-67ff-4740-aa8f-8501300d0604" containerName="kube-rbac-proxy-web"
Jul 21 12:42:38.039519 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:38.039374    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7438cb69-67ff-4740-aa8f-8501300d0604" containerName="kube-rbac-proxy"
Jul 21 12:42:38.039519 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:38.039381    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="7438cb69-67ff-4740-aa8f-8501300d0604" containerName="kube-rbac-proxy"
Jul 21 12:42:38.039519 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:38.039436    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="7438cb69-67ff-4740-aa8f-8501300d0604" containerName="alertmanager"
Jul 21 12:42:38.039519 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:38.039442    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="7438cb69-67ff-4740-aa8f-8501300d0604" containerName="kube-rbac-proxy-metric"
Jul 21 12:42:38.039519 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:38.039446    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="7438cb69-67ff-4740-aa8f-8501300d0604" containerName="kube-rbac-proxy"
Jul 21 12:42:38.039519 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:38.039452    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="8a82abb6-3736-4955-9a1a-21a2b8c9383c" containerName="console"
Jul 21 12:42:38.039519 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:38.039458    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="7438cb69-67ff-4740-aa8f-8501300d0604" containerName="config-reloader"
Jul 21 12:42:38.039519 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:38.039465    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="7438cb69-67ff-4740-aa8f-8501300d0604" containerName="kube-rbac-proxy-web"
Jul 21 12:42:38.039519 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:38.039473    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="7438cb69-67ff-4740-aa8f-8501300d0604" containerName="prom-label-proxy"
Jul 21 12:42:38.044534 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:38.044518    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:42:38.049941 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:38.049897    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 21 12:42:38.049941 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:38.049897    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 21 12:42:38.050138 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:38.049998    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 21 12:42:38.050138 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:38.050000    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 21 12:42:38.050138 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:38.050121    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-kbkl6\""
Jul 21 12:42:38.050289 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:38.050157    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 21 12:42:38.050289 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:38.050184    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 21 12:42:38.050289 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:38.050246    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 21 12:42:38.050289 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:38.050260    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 21 12:42:38.053696 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:38.053679    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 21 12:42:38.056911 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:38.056892    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 21 12:42:38.173055 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:38.172976    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/2daab208-c37f-4c4e-bebd-516f511f318d-config-volume\") pod \"alertmanager-main-0\" (UID: \"2daab208-c37f-4c4e-bebd-516f511f318d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:42:38.173055 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:38.173032    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/2daab208-c37f-4c4e-bebd-516f511f318d-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"2daab208-c37f-4c4e-bebd-516f511f318d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:42:38.173197 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:38.173059    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/2daab208-c37f-4c4e-bebd-516f511f318d-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"2daab208-c37f-4c4e-bebd-516f511f318d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:42:38.173197 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:38.173125    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2daab208-c37f-4c4e-bebd-516f511f318d-config-out\") pod \"alertmanager-main-0\" (UID: \"2daab208-c37f-4c4e-bebd-516f511f318d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:42:38.173197 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:38.173158    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2daab208-c37f-4c4e-bebd-516f511f318d-web-config\") pod \"alertmanager-main-0\" (UID: \"2daab208-c37f-4c4e-bebd-516f511f318d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:42:38.173197 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:38.173186    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2daab208-c37f-4c4e-bebd-516f511f318d-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"2daab208-c37f-4c4e-bebd-516f511f318d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:42:38.173374 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:38.173211    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2daab208-c37f-4c4e-bebd-516f511f318d-tls-assets\") pod \"alertmanager-main-0\" (UID: \"2daab208-c37f-4c4e-bebd-516f511f318d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:42:38.173374 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:38.173243    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2daab208-c37f-4c4e-bebd-516f511f318d-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"2daab208-c37f-4c4e-bebd-516f511f318d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:42:38.173374 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:38.173267    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/2daab208-c37f-4c4e-bebd-516f511f318d-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"2daab208-c37f-4c4e-bebd-516f511f318d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:42:38.173374 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:38.173292    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2xwwz\" (UniqueName: \"kubernetes.io/projected/2daab208-c37f-4c4e-bebd-516f511f318d-kube-api-access-2xwwz\") pod \"alertmanager-main-0\" (UID: \"2daab208-c37f-4c4e-bebd-516f511f318d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:42:38.173374 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:38.173321    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2daab208-c37f-4c4e-bebd-516f511f318d-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"2daab208-c37f-4c4e-bebd-516f511f318d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:42:38.173374 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:38.173353    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/2daab208-c37f-4c4e-bebd-516f511f318d-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"2daab208-c37f-4c4e-bebd-516f511f318d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:42:38.173545 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:38.173404    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2daab208-c37f-4c4e-bebd-516f511f318d-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"2daab208-c37f-4c4e-bebd-516f511f318d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:42:38.273834 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:38.273809    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2daab208-c37f-4c4e-bebd-516f511f318d-config-out\") pod \"alertmanager-main-0\" (UID: \"2daab208-c37f-4c4e-bebd-516f511f318d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:42:38.273934 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:38.273843    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2daab208-c37f-4c4e-bebd-516f511f318d-web-config\") pod \"alertmanager-main-0\" (UID: \"2daab208-c37f-4c4e-bebd-516f511f318d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:42:38.273934 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:38.273869    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2daab208-c37f-4c4e-bebd-516f511f318d-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"2daab208-c37f-4c4e-bebd-516f511f318d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:42:38.273934 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:38.273893    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2daab208-c37f-4c4e-bebd-516f511f318d-tls-assets\") pod \"alertmanager-main-0\" (UID: \"2daab208-c37f-4c4e-bebd-516f511f318d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:42:38.274143 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:38.273932    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2daab208-c37f-4c4e-bebd-516f511f318d-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"2daab208-c37f-4c4e-bebd-516f511f318d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:42:38.274143 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:38.273954    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/2daab208-c37f-4c4e-bebd-516f511f318d-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"2daab208-c37f-4c4e-bebd-516f511f318d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:42:38.274143 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:38.273999    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2xwwz\" (UniqueName: \"kubernetes.io/projected/2daab208-c37f-4c4e-bebd-516f511f318d-kube-api-access-2xwwz\") pod \"alertmanager-main-0\" (UID: \"2daab208-c37f-4c4e-bebd-516f511f318d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:42:38.274143 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:38.274050    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2daab208-c37f-4c4e-bebd-516f511f318d-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"2daab208-c37f-4c4e-bebd-516f511f318d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:42:38.274143 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:38.274106    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/2daab208-c37f-4c4e-bebd-516f511f318d-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"2daab208-c37f-4c4e-bebd-516f511f318d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:42:38.274392 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:38.274158    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2daab208-c37f-4c4e-bebd-516f511f318d-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"2daab208-c37f-4c4e-bebd-516f511f318d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:42:38.274392 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:38.274204    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/2daab208-c37f-4c4e-bebd-516f511f318d-config-volume\") pod \"alertmanager-main-0\" (UID: \"2daab208-c37f-4c4e-bebd-516f511f318d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:42:38.274392 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:38.274233    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/2daab208-c37f-4c4e-bebd-516f511f318d-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"2daab208-c37f-4c4e-bebd-516f511f318d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:42:38.274392 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:38.274258    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/2daab208-c37f-4c4e-bebd-516f511f318d-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"2daab208-c37f-4c4e-bebd-516f511f318d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:42:38.274589 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:38.274552    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/2daab208-c37f-4c4e-bebd-516f511f318d-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"2daab208-c37f-4c4e-bebd-516f511f318d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:42:38.275062 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:38.274994    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2daab208-c37f-4c4e-bebd-516f511f318d-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"2daab208-c37f-4c4e-bebd-516f511f318d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:42:38.275368 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:38.275344    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2daab208-c37f-4c4e-bebd-516f511f318d-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"2daab208-c37f-4c4e-bebd-516f511f318d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:42:38.277061 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:38.277037    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2daab208-c37f-4c4e-bebd-516f511f318d-config-out\") pod \"alertmanager-main-0\" (UID: \"2daab208-c37f-4c4e-bebd-516f511f318d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:42:38.277154 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:38.277097    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2daab208-c37f-4c4e-bebd-516f511f318d-web-config\") pod \"alertmanager-main-0\" (UID: \"2daab208-c37f-4c4e-bebd-516f511f318d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:42:38.277240 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:38.277212    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2daab208-c37f-4c4e-bebd-516f511f318d-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"2daab208-c37f-4c4e-bebd-516f511f318d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:42:38.277302 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:38.277247    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2daab208-c37f-4c4e-bebd-516f511f318d-tls-assets\") pod \"alertmanager-main-0\" (UID: \"2daab208-c37f-4c4e-bebd-516f511f318d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:42:38.277642 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:38.277618    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/2daab208-c37f-4c4e-bebd-516f511f318d-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"2daab208-c37f-4c4e-bebd-516f511f318d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:42:38.277713 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:38.277668    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/2daab208-c37f-4c4e-bebd-516f511f318d-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"2daab208-c37f-4c4e-bebd-516f511f318d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:42:38.277866 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:38.277850    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/2daab208-c37f-4c4e-bebd-516f511f318d-config-volume\") pod \"alertmanager-main-0\" (UID: \"2daab208-c37f-4c4e-bebd-516f511f318d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:42:38.278030 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:38.277989    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2daab208-c37f-4c4e-bebd-516f511f318d-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"2daab208-c37f-4c4e-bebd-516f511f318d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:42:38.279328 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:38.279301    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/2daab208-c37f-4c4e-bebd-516f511f318d-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"2daab208-c37f-4c4e-bebd-516f511f318d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:42:38.283834 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:38.283812    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2xwwz\" (UniqueName: \"kubernetes.io/projected/2daab208-c37f-4c4e-bebd-516f511f318d-kube-api-access-2xwwz\") pod \"alertmanager-main-0\" (UID: \"2daab208-c37f-4c4e-bebd-516f511f318d\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:42:38.359150 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:38.359128    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 21 12:42:38.495032 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:38.494990    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 21 12:42:38.498108 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:42:38.498079    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2daab208_c37f_4c4e_bebd_516f511f318d.slice/crio-097a3beca5400c51cd4d022862eefb72b6f7ebe0434e5c35852ac3e16edc2671 WatchSource:0}: Error finding container 097a3beca5400c51cd4d022862eefb72b6f7ebe0434e5c35852ac3e16edc2671: Status 404 returned error can't find the container with id 097a3beca5400c51cd4d022862eefb72b6f7ebe0434e5c35852ac3e16edc2671
Jul 21 12:42:38.982090 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:38.982061    2556 generic.go:356] "Generic (PLEG): container finished" podID="2daab208-c37f-4c4e-bebd-516f511f318d" containerID="14e1a78adb44067d3abc9f21c94e18f0d3d420c99cf96e71dfa2bba1504730eb" exitCode=0
Jul 21 12:42:38.982426 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:38.982146    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2daab208-c37f-4c4e-bebd-516f511f318d","Type":"ContainerDied","Data":"14e1a78adb44067d3abc9f21c94e18f0d3d420c99cf96e71dfa2bba1504730eb"}
Jul 21 12:42:38.982426 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:38.982210    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2daab208-c37f-4c4e-bebd-516f511f318d","Type":"ContainerStarted","Data":"097a3beca5400c51cd4d022862eefb72b6f7ebe0434e5c35852ac3e16edc2671"}
Jul 21 12:42:39.192740 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:39.192714    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7438cb69-67ff-4740-aa8f-8501300d0604" path="/var/lib/kubelet/pods/7438cb69-67ff-4740-aa8f-8501300d0604/volumes"
Jul 21 12:42:39.818922 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:39.818890    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-867bd765cb-6957j"]
Jul 21 12:42:39.822468 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:39.822448    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-867bd765cb-6957j"
Jul 21 12:42:39.824725 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:39.824706    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Jul 21 12:42:39.825275 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:39.825258    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Jul 21 12:42:39.825275 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:39.825272    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Jul 21 12:42:39.825399 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:39.825261    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-sdblv\""
Jul 21 12:42:39.825601 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:39.825584    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Jul 21 12:42:39.825647 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:39.825584    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Jul 21 12:42:39.831485 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:39.831468    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Jul 21 12:42:39.835775 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:39.835756    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-867bd765cb-6957j"]
Jul 21 12:42:39.988804 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:39.988778    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/ef6b7630-4103-4b25-82e2-249ce1c9f05f-telemeter-client-tls\") pod \"telemeter-client-867bd765cb-6957j\" (UID: \"ef6b7630-4103-4b25-82e2-249ce1c9f05f\") " pod="openshift-monitoring/telemeter-client-867bd765cb-6957j"
Jul 21 12:42:39.989188 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:39.988828    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/ef6b7630-4103-4b25-82e2-249ce1c9f05f-secret-telemeter-client\") pod \"telemeter-client-867bd765cb-6957j\" (UID: \"ef6b7630-4103-4b25-82e2-249ce1c9f05f\") " pod="openshift-monitoring/telemeter-client-867bd765cb-6957j"
Jul 21 12:42:39.989188 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:39.988872    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tgmk9\" (UniqueName: \"kubernetes.io/projected/ef6b7630-4103-4b25-82e2-249ce1c9f05f-kube-api-access-tgmk9\") pod \"telemeter-client-867bd765cb-6957j\" (UID: \"ef6b7630-4103-4b25-82e2-249ce1c9f05f\") " pod="openshift-monitoring/telemeter-client-867bd765cb-6957j"
Jul 21 12:42:39.989188 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:39.988908    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ef6b7630-4103-4b25-82e2-249ce1c9f05f-telemeter-trusted-ca-bundle\") pod \"telemeter-client-867bd765cb-6957j\" (UID: \"ef6b7630-4103-4b25-82e2-249ce1c9f05f\") " pod="openshift-monitoring/telemeter-client-867bd765cb-6957j"
Jul 21 12:42:39.989188 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:39.988942    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ef6b7630-4103-4b25-82e2-249ce1c9f05f-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-867bd765cb-6957j\" (UID: \"ef6b7630-4103-4b25-82e2-249ce1c9f05f\") " pod="openshift-monitoring/telemeter-client-867bd765cb-6957j"
Jul 21 12:42:39.989188 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:39.988974    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/ef6b7630-4103-4b25-82e2-249ce1c9f05f-federate-client-tls\") pod \"telemeter-client-867bd765cb-6957j\" (UID: \"ef6b7630-4103-4b25-82e2-249ce1c9f05f\") " pod="openshift-monitoring/telemeter-client-867bd765cb-6957j"
Jul 21 12:42:39.989188 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:39.989054    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ef6b7630-4103-4b25-82e2-249ce1c9f05f-metrics-client-ca\") pod \"telemeter-client-867bd765cb-6957j\" (UID: \"ef6b7630-4103-4b25-82e2-249ce1c9f05f\") " pod="openshift-monitoring/telemeter-client-867bd765cb-6957j"
Jul 21 12:42:39.989188 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:39.989096    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ef6b7630-4103-4b25-82e2-249ce1c9f05f-serving-certs-ca-bundle\") pod \"telemeter-client-867bd765cb-6957j\" (UID: \"ef6b7630-4103-4b25-82e2-249ce1c9f05f\") " pod="openshift-monitoring/telemeter-client-867bd765cb-6957j"
Jul 21 12:42:39.994609 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:39.994580    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2daab208-c37f-4c4e-bebd-516f511f318d","Type":"ContainerStarted","Data":"2de17e97e9b446395cbfb6c3ad54b16c40e048217f45bed80c93a405898616f1"}
Jul 21 12:42:39.994702 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:39.994680    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2daab208-c37f-4c4e-bebd-516f511f318d","Type":"ContainerStarted","Data":"f5900e67f972a663c44ed22c450c247a952d1afbe9421a792c5991b702b6a8b8"}
Jul 21 12:42:39.994737 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:39.994708    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2daab208-c37f-4c4e-bebd-516f511f318d","Type":"ContainerStarted","Data":"aa50bbb8b342879ae5abf5fcbc27f3ffdecb0994c456a8a4413b8a28e3fdd034"}
Jul 21 12:42:39.994775 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:39.994736    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2daab208-c37f-4c4e-bebd-516f511f318d","Type":"ContainerStarted","Data":"d2802018ba6ce0d1f1039e0141487681b34c78067231b048869112848b723d8d"}
Jul 21 12:42:39.994806 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:39.994776    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2daab208-c37f-4c4e-bebd-516f511f318d","Type":"ContainerStarted","Data":"35ba24fa51a78bd9cca64bc87bb5fb66eb9aab082f4ac10b0eb26b43f4e933e4"}
Jul 21 12:42:39.994837 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:39.994804    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2daab208-c37f-4c4e-bebd-516f511f318d","Type":"ContainerStarted","Data":"8503cc4e69792772e88ef95048cde1cb52a2f53f5dcea83e22f73215a8508ec0"}
Jul 21 12:42:40.026668 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:40.026624    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.010409658 podStartE2EDuration="2.026612752s" totalImagesPullingTime="0s" totalInitContainerRuntime="16.203094ms" isStatefulPod=false podCreationTimestamp="2026-07-21 12:42:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:42:40.025252682 +0000 UTC m=+173.329257770" watchObservedRunningTime="2026-07-21 12:42:40.026612752 +0000 UTC m=+173.330617892"
Jul 21 12:42:40.090444 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:40.090388    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tgmk9\" (UniqueName: \"kubernetes.io/projected/ef6b7630-4103-4b25-82e2-249ce1c9f05f-kube-api-access-tgmk9\") pod \"telemeter-client-867bd765cb-6957j\" (UID: \"ef6b7630-4103-4b25-82e2-249ce1c9f05f\") " pod="openshift-monitoring/telemeter-client-867bd765cb-6957j"
Jul 21 12:42:40.090444 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:40.090417    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ef6b7630-4103-4b25-82e2-249ce1c9f05f-telemeter-trusted-ca-bundle\") pod \"telemeter-client-867bd765cb-6957j\" (UID: \"ef6b7630-4103-4b25-82e2-249ce1c9f05f\") " pod="openshift-monitoring/telemeter-client-867bd765cb-6957j"
Jul 21 12:42:40.090444 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:40.090436    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ef6b7630-4103-4b25-82e2-249ce1c9f05f-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-867bd765cb-6957j\" (UID: \"ef6b7630-4103-4b25-82e2-249ce1c9f05f\") " pod="openshift-monitoring/telemeter-client-867bd765cb-6957j"
Jul 21 12:42:40.090637 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:40.090476    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/ef6b7630-4103-4b25-82e2-249ce1c9f05f-federate-client-tls\") pod \"telemeter-client-867bd765cb-6957j\" (UID: \"ef6b7630-4103-4b25-82e2-249ce1c9f05f\") " pod="openshift-monitoring/telemeter-client-867bd765cb-6957j"
Jul 21 12:42:40.090637 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:40.090573    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ef6b7630-4103-4b25-82e2-249ce1c9f05f-metrics-client-ca\") pod \"telemeter-client-867bd765cb-6957j\" (UID: \"ef6b7630-4103-4b25-82e2-249ce1c9f05f\") " pod="openshift-monitoring/telemeter-client-867bd765cb-6957j"
Jul 21 12:42:40.090737 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:40.090631    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ef6b7630-4103-4b25-82e2-249ce1c9f05f-serving-certs-ca-bundle\") pod \"telemeter-client-867bd765cb-6957j\" (UID: \"ef6b7630-4103-4b25-82e2-249ce1c9f05f\") " pod="openshift-monitoring/telemeter-client-867bd765cb-6957j"
Jul 21 12:42:40.090737 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:40.090722    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/ef6b7630-4103-4b25-82e2-249ce1c9f05f-telemeter-client-tls\") pod \"telemeter-client-867bd765cb-6957j\" (UID: \"ef6b7630-4103-4b25-82e2-249ce1c9f05f\") " pod="openshift-monitoring/telemeter-client-867bd765cb-6957j"
Jul 21 12:42:40.090832 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:40.090789    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/ef6b7630-4103-4b25-82e2-249ce1c9f05f-secret-telemeter-client\") pod \"telemeter-client-867bd765cb-6957j\" (UID: \"ef6b7630-4103-4b25-82e2-249ce1c9f05f\") " pod="openshift-monitoring/telemeter-client-867bd765cb-6957j"
Jul 21 12:42:40.091569 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:40.091540    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ef6b7630-4103-4b25-82e2-249ce1c9f05f-serving-certs-ca-bundle\") pod \"telemeter-client-867bd765cb-6957j\" (UID: \"ef6b7630-4103-4b25-82e2-249ce1c9f05f\") " pod="openshift-monitoring/telemeter-client-867bd765cb-6957j"
Jul 21 12:42:40.091662 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:40.091578    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ef6b7630-4103-4b25-82e2-249ce1c9f05f-metrics-client-ca\") pod \"telemeter-client-867bd765cb-6957j\" (UID: \"ef6b7630-4103-4b25-82e2-249ce1c9f05f\") " pod="openshift-monitoring/telemeter-client-867bd765cb-6957j"
Jul 21 12:42:40.091662 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:40.091584    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ef6b7630-4103-4b25-82e2-249ce1c9f05f-telemeter-trusted-ca-bundle\") pod \"telemeter-client-867bd765cb-6957j\" (UID: \"ef6b7630-4103-4b25-82e2-249ce1c9f05f\") " pod="openshift-monitoring/telemeter-client-867bd765cb-6957j"
Jul 21 12:42:40.093238 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:40.093218    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/ef6b7630-4103-4b25-82e2-249ce1c9f05f-telemeter-client-tls\") pod \"telemeter-client-867bd765cb-6957j\" (UID: \"ef6b7630-4103-4b25-82e2-249ce1c9f05f\") " pod="openshift-monitoring/telemeter-client-867bd765cb-6957j"
Jul 21 12:42:40.093556 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:40.093537    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ef6b7630-4103-4b25-82e2-249ce1c9f05f-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-867bd765cb-6957j\" (UID: \"ef6b7630-4103-4b25-82e2-249ce1c9f05f\") " pod="openshift-monitoring/telemeter-client-867bd765cb-6957j"
Jul 21 12:42:40.093641 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:40.093630    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/ef6b7630-4103-4b25-82e2-249ce1c9f05f-secret-telemeter-client\") pod \"telemeter-client-867bd765cb-6957j\" (UID: \"ef6b7630-4103-4b25-82e2-249ce1c9f05f\") " pod="openshift-monitoring/telemeter-client-867bd765cb-6957j"
Jul 21 12:42:40.093690 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:40.093635    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/ef6b7630-4103-4b25-82e2-249ce1c9f05f-federate-client-tls\") pod \"telemeter-client-867bd765cb-6957j\" (UID: \"ef6b7630-4103-4b25-82e2-249ce1c9f05f\") " pod="openshift-monitoring/telemeter-client-867bd765cb-6957j"
Jul 21 12:42:40.098597 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:40.098580    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tgmk9\" (UniqueName: \"kubernetes.io/projected/ef6b7630-4103-4b25-82e2-249ce1c9f05f-kube-api-access-tgmk9\") pod \"telemeter-client-867bd765cb-6957j\" (UID: \"ef6b7630-4103-4b25-82e2-249ce1c9f05f\") " pod="openshift-monitoring/telemeter-client-867bd765cb-6957j"
Jul 21 12:42:40.131605 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:40.131585    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-867bd765cb-6957j"
Jul 21 12:42:40.274096 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:40.274065    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-867bd765cb-6957j"]
Jul 21 12:42:40.276631 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:42:40.276606    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef6b7630_4103_4b25_82e2_249ce1c9f05f.slice/crio-68fd8a331cb2898b6384dba9e329adb5ab5f0cd674bcb0c58cc87d00e2a0fa0b WatchSource:0}: Error finding container 68fd8a331cb2898b6384dba9e329adb5ab5f0cd674bcb0c58cc87d00e2a0fa0b: Status 404 returned error can't find the container with id 68fd8a331cb2898b6384dba9e329adb5ab5f0cd674bcb0c58cc87d00e2a0fa0b
Jul 21 12:42:41.001158 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:41.001127    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-867bd765cb-6957j" event={"ID":"ef6b7630-4103-4b25-82e2-249ce1c9f05f","Type":"ContainerStarted","Data":"68fd8a331cb2898b6384dba9e329adb5ab5f0cd674bcb0c58cc87d00e2a0fa0b"}
Jul 21 12:42:43.010630 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:43.010599    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-867bd765cb-6957j" event={"ID":"ef6b7630-4103-4b25-82e2-249ce1c9f05f","Type":"ContainerStarted","Data":"288faedffa7873d799251a6622524d73463dbf60477f72ec31e867213c9457ce"}
Jul 21 12:42:43.010971 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:43.010651    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-867bd765cb-6957j" event={"ID":"ef6b7630-4103-4b25-82e2-249ce1c9f05f","Type":"ContainerStarted","Data":"2f630f481b4dc6615e0887de7886b0ec8ec9794a90cddeb90cc3e3f9acfa30d9"}
Jul 21 12:42:43.010971 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:43.010670    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-867bd765cb-6957j" event={"ID":"ef6b7630-4103-4b25-82e2-249ce1c9f05f","Type":"ContainerStarted","Data":"413753ab4ae340aa4716ae8f79bf4432c1d2dea79540e8a7e25ec8fa7e2ef8bf"}
Jul 21 12:42:43.035726 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:42:43.035668    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-867bd765cb-6957j" podStartSLOduration=2.195530259 podStartE2EDuration="4.035648338s" totalImagesPullingTime="1.840118079s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:42:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:42:43.033887051 +0000 UTC m=+176.337892138" watchObservedRunningTime="2026-07-21 12:42:43.035648338 +0000 UTC m=+176.339653428"
Jul 21 12:44:47.075195 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:44:47.075163    2556 kubelet.go:1648] "Image garbage collection succeeded"
Jul 21 12:45:53.219256 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:45:53.219227    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-bb5rg"]
Jul 21 12:45:53.222351 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:45:53.222334    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-bb5rg"
Jul 21 12:45:53.224836 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:45:53.224813    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 21 12:45:53.225346 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:45:53.225327    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 21 12:45:53.225346 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:45:53.225342    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-jxskx\""
Jul 21 12:45:53.233440 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:45:53.233418    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-bb5rg"]
Jul 21 12:45:53.265112 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:45:53.265088    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9wpd4\" (UniqueName: \"kubernetes.io/projected/caaa9662-60e4-4ffd-a8a1-d70b1af91b6d-kube-api-access-9wpd4\") pod \"cert-manager-cainjector-559cdd8c87-bb5rg\" (UID: \"caaa9662-60e4-4ffd-a8a1-d70b1af91b6d\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-bb5rg"
Jul 21 12:45:53.265202 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:45:53.265144    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/caaa9662-60e4-4ffd-a8a1-d70b1af91b6d-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-bb5rg\" (UID: \"caaa9662-60e4-4ffd-a8a1-d70b1af91b6d\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-bb5rg"
Jul 21 12:45:53.365690 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:45:53.365669    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/caaa9662-60e4-4ffd-a8a1-d70b1af91b6d-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-bb5rg\" (UID: \"caaa9662-60e4-4ffd-a8a1-d70b1af91b6d\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-bb5rg"
Jul 21 12:45:53.365774 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:45:53.365710    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9wpd4\" (UniqueName: \"kubernetes.io/projected/caaa9662-60e4-4ffd-a8a1-d70b1af91b6d-kube-api-access-9wpd4\") pod \"cert-manager-cainjector-559cdd8c87-bb5rg\" (UID: \"caaa9662-60e4-4ffd-a8a1-d70b1af91b6d\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-bb5rg"
Jul 21 12:45:53.375282 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:45:53.375257    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9wpd4\" (UniqueName: \"kubernetes.io/projected/caaa9662-60e4-4ffd-a8a1-d70b1af91b6d-kube-api-access-9wpd4\") pod \"cert-manager-cainjector-559cdd8c87-bb5rg\" (UID: \"caaa9662-60e4-4ffd-a8a1-d70b1af91b6d\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-bb5rg"
Jul 21 12:45:53.375436 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:45:53.375419    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/caaa9662-60e4-4ffd-a8a1-d70b1af91b6d-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-bb5rg\" (UID: \"caaa9662-60e4-4ffd-a8a1-d70b1af91b6d\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-bb5rg"
Jul 21 12:45:53.530714 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:45:53.530692    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-bb5rg"
Jul 21 12:45:53.655347 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:45:53.655325    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-bb5rg"]
Jul 21 12:45:53.657949 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:45:53.657920    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcaaa9662_60e4_4ffd_a8a1_d70b1af91b6d.slice/crio-a5422fee2eda70d7db1ef316adcab5183f862816c1c84b854330394c799e3a33 WatchSource:0}: Error finding container a5422fee2eda70d7db1ef316adcab5183f862816c1c84b854330394c799e3a33: Status 404 returned error can't find the container with id a5422fee2eda70d7db1ef316adcab5183f862816c1c84b854330394c799e3a33
Jul 21 12:45:53.660133 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:45:53.660108    2556 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 21 12:45:53.742750 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:45:53.742721    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-bb5rg" event={"ID":"caaa9662-60e4-4ffd-a8a1-d70b1af91b6d","Type":"ContainerStarted","Data":"a5422fee2eda70d7db1ef316adcab5183f862816c1c84b854330394c799e3a33"}
Jul 21 12:45:57.765527 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:45:57.765488    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-bb5rg" event={"ID":"caaa9662-60e4-4ffd-a8a1-d70b1af91b6d","Type":"ContainerStarted","Data":"d4dbb29eed27707c0b4cd8fb553100d8ca8b1605c748e4c915f40d476e2f1c20"}
Jul 21 12:45:57.781856 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:45:57.781796    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-bb5rg" podStartSLOduration=1.690954535 podStartE2EDuration="4.781778439s" totalImagesPullingTime="3.090823904s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:45:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:45:57.781335643 +0000 UTC m=+371.085340732" watchObservedRunningTime="2026-07-21 12:45:57.781778439 +0000 UTC m=+371.085783531"
Jul 21 12:46:10.547706 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:46:10.547663    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-xhrm6"]
Jul 21 12:46:10.552123 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:46:10.552098    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-xhrm6"
Jul 21 12:46:10.554462 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:46:10.554436    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 21 12:46:10.554462 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:46:10.554449    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 21 12:46:10.555162 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:46:10.555149    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-2dc26\""
Jul 21 12:46:10.559765 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:46:10.559747    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-xhrm6"]
Jul 21 12:46:10.609415 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:46:10.609388    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qhltm\" (UniqueName: \"kubernetes.io/projected/b3312e1f-054d-4e08-863b-bf3c5dd54a73-kube-api-access-qhltm\") pod \"openshift-lws-operator-fd8ccff4c-xhrm6\" (UID: \"b3312e1f-054d-4e08-863b-bf3c5dd54a73\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-xhrm6"
Jul 21 12:46:10.609561 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:46:10.609439    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b3312e1f-054d-4e08-863b-bf3c5dd54a73-tmp\") pod \"openshift-lws-operator-fd8ccff4c-xhrm6\" (UID: \"b3312e1f-054d-4e08-863b-bf3c5dd54a73\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-xhrm6"
Jul 21 12:46:10.710650 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:46:10.710617    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b3312e1f-054d-4e08-863b-bf3c5dd54a73-tmp\") pod \"openshift-lws-operator-fd8ccff4c-xhrm6\" (UID: \"b3312e1f-054d-4e08-863b-bf3c5dd54a73\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-xhrm6"
Jul 21 12:46:10.710806 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:46:10.710723    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qhltm\" (UniqueName: \"kubernetes.io/projected/b3312e1f-054d-4e08-863b-bf3c5dd54a73-kube-api-access-qhltm\") pod \"openshift-lws-operator-fd8ccff4c-xhrm6\" (UID: \"b3312e1f-054d-4e08-863b-bf3c5dd54a73\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-xhrm6"
Jul 21 12:46:10.711001 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:46:10.710981    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b3312e1f-054d-4e08-863b-bf3c5dd54a73-tmp\") pod \"openshift-lws-operator-fd8ccff4c-xhrm6\" (UID: \"b3312e1f-054d-4e08-863b-bf3c5dd54a73\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-xhrm6"
Jul 21 12:46:10.719973 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:46:10.719950    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qhltm\" (UniqueName: \"kubernetes.io/projected/b3312e1f-054d-4e08-863b-bf3c5dd54a73-kube-api-access-qhltm\") pod \"openshift-lws-operator-fd8ccff4c-xhrm6\" (UID: \"b3312e1f-054d-4e08-863b-bf3c5dd54a73\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-xhrm6"
Jul 21 12:46:10.861333 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:46:10.861250    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-xhrm6"
Jul 21 12:46:10.994929 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:46:10.994904    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-xhrm6"]
Jul 21 12:46:10.997470 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:46:10.997441    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb3312e1f_054d_4e08_863b_bf3c5dd54a73.slice/crio-8125c2d61db00f1f597c84206d9a6884aca0516c99ff1d118c25637acf0e5367 WatchSource:0}: Error finding container 8125c2d61db00f1f597c84206d9a6884aca0516c99ff1d118c25637acf0e5367: Status 404 returned error can't find the container with id 8125c2d61db00f1f597c84206d9a6884aca0516c99ff1d118c25637acf0e5367
Jul 21 12:46:11.830134 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:46:11.830087    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-xhrm6" event={"ID":"b3312e1f-054d-4e08-863b-bf3c5dd54a73","Type":"ContainerStarted","Data":"8125c2d61db00f1f597c84206d9a6884aca0516c99ff1d118c25637acf0e5367"}
Jul 21 12:46:13.841515 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:46:13.841478    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-xhrm6" event={"ID":"b3312e1f-054d-4e08-863b-bf3c5dd54a73","Type":"ContainerStarted","Data":"dee61492957df9393f8d9684e958e4fb919e9f95883f0cbc0361ec91a08a1ef1"}
Jul 21 12:46:13.858724 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:46:13.858665    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-xhrm6" podStartSLOduration=1.391824385 podStartE2EDuration="3.85864586s" totalImagesPullingTime="2.466821475s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:46:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:46:13.857583416 +0000 UTC m=+387.161588505" watchObservedRunningTime="2026-07-21 12:46:13.85864586 +0000 UTC m=+387.162650952"
Jul 21 12:46:32.720003 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:46:32.719962    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-579d89bddd-trnmc"]
Jul 21 12:46:32.726921 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:46:32.726892    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-579d89bddd-trnmc"
Jul 21 12:46:32.729374 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:46:32.729351    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 21 12:46:32.729374 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:46:32.729359    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 21 12:46:32.729955 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:46:32.729925    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 21 12:46:32.729955 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:46:32.729937    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 21 12:46:32.729955 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:46:32.729940    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-bh5f5\""
Jul 21 12:46:32.743656 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:46:32.743633    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-579d89bddd-trnmc"]
Jul 21 12:46:32.905282 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:46:32.905253    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/226755f7-2596-454a-a055-36be3e543967-manifests\") pod \"opendatahub-operator-controller-manager-579d89bddd-trnmc\" (UID: \"226755f7-2596-454a-a055-36be3e543967\") " pod="opendatahub/opendatahub-operator-controller-manager-579d89bddd-trnmc"
Jul 21 12:46:32.905282 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:46:32.905285    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/226755f7-2596-454a-a055-36be3e543967-tmp\") pod \"opendatahub-operator-controller-manager-579d89bddd-trnmc\" (UID: \"226755f7-2596-454a-a055-36be3e543967\") " pod="opendatahub/opendatahub-operator-controller-manager-579d89bddd-trnmc"
Jul 21 12:46:32.905524 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:46:32.905331    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8j9ss\" (UniqueName: \"kubernetes.io/projected/226755f7-2596-454a-a055-36be3e543967-kube-api-access-8j9ss\") pod \"opendatahub-operator-controller-manager-579d89bddd-trnmc\" (UID: \"226755f7-2596-454a-a055-36be3e543967\") " pod="opendatahub/opendatahub-operator-controller-manager-579d89bddd-trnmc"
Jul 21 12:46:32.905524 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:46:32.905352    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/226755f7-2596-454a-a055-36be3e543967-apiservice-cert\") pod \"opendatahub-operator-controller-manager-579d89bddd-trnmc\" (UID: \"226755f7-2596-454a-a055-36be3e543967\") " pod="opendatahub/opendatahub-operator-controller-manager-579d89bddd-trnmc"
Jul 21 12:46:32.905524 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:46:32.905369    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/226755f7-2596-454a-a055-36be3e543967-webhook-cert\") pod \"opendatahub-operator-controller-manager-579d89bddd-trnmc\" (UID: \"226755f7-2596-454a-a055-36be3e543967\") " pod="opendatahub/opendatahub-operator-controller-manager-579d89bddd-trnmc"
Jul 21 12:46:33.006832 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:46:33.006794    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/226755f7-2596-454a-a055-36be3e543967-manifests\") pod \"opendatahub-operator-controller-manager-579d89bddd-trnmc\" (UID: \"226755f7-2596-454a-a055-36be3e543967\") " pod="opendatahub/opendatahub-operator-controller-manager-579d89bddd-trnmc"
Jul 21 12:46:33.006832 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:46:33.006837    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/226755f7-2596-454a-a055-36be3e543967-tmp\") pod \"opendatahub-operator-controller-manager-579d89bddd-trnmc\" (UID: \"226755f7-2596-454a-a055-36be3e543967\") " pod="opendatahub/opendatahub-operator-controller-manager-579d89bddd-trnmc"
Jul 21 12:46:33.007105 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:46:33.006890    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8j9ss\" (UniqueName: \"kubernetes.io/projected/226755f7-2596-454a-a055-36be3e543967-kube-api-access-8j9ss\") pod \"opendatahub-operator-controller-manager-579d89bddd-trnmc\" (UID: \"226755f7-2596-454a-a055-36be3e543967\") " pod="opendatahub/opendatahub-operator-controller-manager-579d89bddd-trnmc"
Jul 21 12:46:33.007105 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:46:33.006911    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/226755f7-2596-454a-a055-36be3e543967-apiservice-cert\") pod \"opendatahub-operator-controller-manager-579d89bddd-trnmc\" (UID: \"226755f7-2596-454a-a055-36be3e543967\") " pod="opendatahub/opendatahub-operator-controller-manager-579d89bddd-trnmc"
Jul 21 12:46:33.007105 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:46:33.006925    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/226755f7-2596-454a-a055-36be3e543967-webhook-cert\") pod \"opendatahub-operator-controller-manager-579d89bddd-trnmc\" (UID: \"226755f7-2596-454a-a055-36be3e543967\") " pod="opendatahub/opendatahub-operator-controller-manager-579d89bddd-trnmc"
Jul 21 12:46:33.007329 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:46:33.007306    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/226755f7-2596-454a-a055-36be3e543967-manifests\") pod \"opendatahub-operator-controller-manager-579d89bddd-trnmc\" (UID: \"226755f7-2596-454a-a055-36be3e543967\") " pod="opendatahub/opendatahub-operator-controller-manager-579d89bddd-trnmc"
Jul 21 12:46:33.009606 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:46:33.009577    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/226755f7-2596-454a-a055-36be3e543967-tmp\") pod \"opendatahub-operator-controller-manager-579d89bddd-trnmc\" (UID: \"226755f7-2596-454a-a055-36be3e543967\") " pod="opendatahub/opendatahub-operator-controller-manager-579d89bddd-trnmc"
Jul 21 12:46:33.009736 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:46:33.009709    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/226755f7-2596-454a-a055-36be3e543967-webhook-cert\") pod \"opendatahub-operator-controller-manager-579d89bddd-trnmc\" (UID: \"226755f7-2596-454a-a055-36be3e543967\") " pod="opendatahub/opendatahub-operator-controller-manager-579d89bddd-trnmc"
Jul 21 12:46:33.009799 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:46:33.009712    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/226755f7-2596-454a-a055-36be3e543967-apiservice-cert\") pod \"opendatahub-operator-controller-manager-579d89bddd-trnmc\" (UID: \"226755f7-2596-454a-a055-36be3e543967\") " pod="opendatahub/opendatahub-operator-controller-manager-579d89bddd-trnmc"
Jul 21 12:46:33.015859 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:46:33.015835    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8j9ss\" (UniqueName: \"kubernetes.io/projected/226755f7-2596-454a-a055-36be3e543967-kube-api-access-8j9ss\") pod \"opendatahub-operator-controller-manager-579d89bddd-trnmc\" (UID: \"226755f7-2596-454a-a055-36be3e543967\") " pod="opendatahub/opendatahub-operator-controller-manager-579d89bddd-trnmc"
Jul 21 12:46:33.036911 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:46:33.036883    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-579d89bddd-trnmc"
Jul 21 12:46:33.180522 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:46:33.180488    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-579d89bddd-trnmc"]
Jul 21 12:46:33.185090 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:46:33.185061    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod226755f7_2596_454a_a055_36be3e543967.slice/crio-61462b804048b6d652bb15678e76b654e66fe843cb9d1e4be4cde88e1c2cef92 WatchSource:0}: Error finding container 61462b804048b6d652bb15678e76b654e66fe843cb9d1e4be4cde88e1c2cef92: Status 404 returned error can't find the container with id 61462b804048b6d652bb15678e76b654e66fe843cb9d1e4be4cde88e1c2cef92
Jul 21 12:46:33.928206 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:46:33.928151    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-579d89bddd-trnmc" event={"ID":"226755f7-2596-454a-a055-36be3e543967","Type":"ContainerStarted","Data":"61462b804048b6d652bb15678e76b654e66fe843cb9d1e4be4cde88e1c2cef92"}
Jul 21 12:46:36.944401 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:46:36.944348    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-579d89bddd-trnmc" event={"ID":"226755f7-2596-454a-a055-36be3e543967","Type":"ContainerStarted","Data":"a3e7a5b0c0076fa803faa4aea10ca7a99355c5204f259d95d4767e004367ebe4"}
Jul 21 12:46:37.949943 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:46:37.949906    2556 generic.go:356] "Generic (PLEG): container finished" podID="226755f7-2596-454a-a055-36be3e543967" containerID="a3e7a5b0c0076fa803faa4aea10ca7a99355c5204f259d95d4767e004367ebe4" exitCode=0
Jul 21 12:46:37.950361 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:46:37.949999    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-579d89bddd-trnmc" event={"ID":"226755f7-2596-454a-a055-36be3e543967","Type":"ContainerDied","Data":"a3e7a5b0c0076fa803faa4aea10ca7a99355c5204f259d95d4767e004367ebe4"}
Jul 21 12:46:38.958759 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:46:38.958723    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-579d89bddd-trnmc" event={"ID":"226755f7-2596-454a-a055-36be3e543967","Type":"ContainerStarted","Data":"1d06ddbc881b26d650be58923f0f1fcf0c9e99d11101cfa5fd2155956de20c67"}
Jul 21 12:46:38.959194 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:46:38.958807    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-579d89bddd-trnmc"
Jul 21 12:46:38.983152 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:46:38.983108    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-579d89bddd-trnmc" podStartSLOduration=2.477638629 podStartE2EDuration="6.983090172s" totalImagesPullingTime="3.428717056s" totalInitContainerRuntime="1.076734487s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:46:32 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:46:38.98074906 +0000 UTC m=+412.284754150" watchObservedRunningTime="2026-07-21 12:46:38.983090172 +0000 UTC m=+412.287095261"
Jul 21 12:46:49.964733 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:46:49.964704    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-579d89bddd-trnmc"
Jul 21 12:46:52.295332 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:46:52.295295    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-67cb6d6b5d-rfwlt"]
Jul 21 12:46:52.307041 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:46:52.306989    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-67cb6d6b5d-rfwlt"
Jul 21 12:46:52.308894 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:46:52.308865    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-67cb6d6b5d-rfwlt"]
Jul 21 12:46:52.309263 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:46:52.309246    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 21 12:46:52.310060 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:46:52.310006    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 21 12:46:52.310187 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:46:52.310033    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-kr5rq\""
Jul 21 12:46:52.310187 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:46:52.310128    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 21 12:46:52.310787 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:46:52.310767    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 21 12:46:52.366974 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:46:52.366946    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/041ed91e-0571-4917-a641-c554b8734229-tmp\") pod \"kube-auth-proxy-67cb6d6b5d-rfwlt\" (UID: \"041ed91e-0571-4917-a641-c554b8734229\") " pod="openshift-ingress/kube-auth-proxy-67cb6d6b5d-rfwlt"
Jul 21 12:46:52.367130 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:46:52.366983    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/041ed91e-0571-4917-a641-c554b8734229-tls-certs\") pod \"kube-auth-proxy-67cb6d6b5d-rfwlt\" (UID: \"041ed91e-0571-4917-a641-c554b8734229\") " pod="openshift-ingress/kube-auth-proxy-67cb6d6b5d-rfwlt"
Jul 21 12:46:52.367130 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:46:52.367033    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bkht8\" (UniqueName: \"kubernetes.io/projected/041ed91e-0571-4917-a641-c554b8734229-kube-api-access-bkht8\") pod \"kube-auth-proxy-67cb6d6b5d-rfwlt\" (UID: \"041ed91e-0571-4917-a641-c554b8734229\") " pod="openshift-ingress/kube-auth-proxy-67cb6d6b5d-rfwlt"
Jul 21 12:46:52.429166 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:46:52.429138    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-s7tkd"]
Jul 21 12:46:52.432564 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:46:52.432546    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-s7tkd"
Jul 21 12:46:52.434865 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:46:52.434849    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-2hmh6\""
Jul 21 12:46:52.435128 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:46:52.435108    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Jul 21 12:46:52.450771 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:46:52.450745    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-s7tkd"]
Jul 21 12:46:52.468340 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:46:52.468307    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7bc3e182-acec-4866-9dd0-f3db1e521186-cert\") pod \"odh-model-controller-7998c67dc4-s7tkd\" (UID: \"7bc3e182-acec-4866-9dd0-f3db1e521186\") " pod="opendatahub/odh-model-controller-7998c67dc4-s7tkd"
Jul 21 12:46:52.468340 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:46:52.468340    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-btncj\" (UniqueName: \"kubernetes.io/projected/7bc3e182-acec-4866-9dd0-f3db1e521186-kube-api-access-btncj\") pod \"odh-model-controller-7998c67dc4-s7tkd\" (UID: \"7bc3e182-acec-4866-9dd0-f3db1e521186\") " pod="opendatahub/odh-model-controller-7998c67dc4-s7tkd"
Jul 21 12:46:52.468543 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:46:52.468393    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bkht8\" (UniqueName: \"kubernetes.io/projected/041ed91e-0571-4917-a641-c554b8734229-kube-api-access-bkht8\") pod \"kube-auth-proxy-67cb6d6b5d-rfwlt\" (UID: \"041ed91e-0571-4917-a641-c554b8734229\") " pod="openshift-ingress/kube-auth-proxy-67cb6d6b5d-rfwlt"
Jul 21 12:46:52.468543 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:46:52.468456    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/041ed91e-0571-4917-a641-c554b8734229-tmp\") pod \"kube-auth-proxy-67cb6d6b5d-rfwlt\" (UID: \"041ed91e-0571-4917-a641-c554b8734229\") " pod="openshift-ingress/kube-auth-proxy-67cb6d6b5d-rfwlt"
Jul 21 12:46:52.468543 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:46:52.468483    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/041ed91e-0571-4917-a641-c554b8734229-tls-certs\") pod \"kube-auth-proxy-67cb6d6b5d-rfwlt\" (UID: \"041ed91e-0571-4917-a641-c554b8734229\") " pod="openshift-ingress/kube-auth-proxy-67cb6d6b5d-rfwlt"
Jul 21 12:46:52.470822 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:46:52.470793    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/041ed91e-0571-4917-a641-c554b8734229-tmp\") pod \"kube-auth-proxy-67cb6d6b5d-rfwlt\" (UID: \"041ed91e-0571-4917-a641-c554b8734229\") " pod="openshift-ingress/kube-auth-proxy-67cb6d6b5d-rfwlt"
Jul 21 12:46:52.471113 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:46:52.471096    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/041ed91e-0571-4917-a641-c554b8734229-tls-certs\") pod \"kube-auth-proxy-67cb6d6b5d-rfwlt\" (UID: \"041ed91e-0571-4917-a641-c554b8734229\") " pod="openshift-ingress/kube-auth-proxy-67cb6d6b5d-rfwlt"
Jul 21 12:46:52.479997 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:46:52.479975    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bkht8\" (UniqueName: \"kubernetes.io/projected/041ed91e-0571-4917-a641-c554b8734229-kube-api-access-bkht8\") pod \"kube-auth-proxy-67cb6d6b5d-rfwlt\" (UID: \"041ed91e-0571-4917-a641-c554b8734229\") " pod="openshift-ingress/kube-auth-proxy-67cb6d6b5d-rfwlt"
Jul 21 12:46:52.569205 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:46:52.569133    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7bc3e182-acec-4866-9dd0-f3db1e521186-cert\") pod \"odh-model-controller-7998c67dc4-s7tkd\" (UID: \"7bc3e182-acec-4866-9dd0-f3db1e521186\") " pod="opendatahub/odh-model-controller-7998c67dc4-s7tkd"
Jul 21 12:46:52.569205 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:46:52.569169    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-btncj\" (UniqueName: \"kubernetes.io/projected/7bc3e182-acec-4866-9dd0-f3db1e521186-kube-api-access-btncj\") pod \"odh-model-controller-7998c67dc4-s7tkd\" (UID: \"7bc3e182-acec-4866-9dd0-f3db1e521186\") " pod="opendatahub/odh-model-controller-7998c67dc4-s7tkd"
Jul 21 12:46:52.569358 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:46:52.569272    2556 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Jul 21 12:46:52.569358 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:46:52.569330    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7bc3e182-acec-4866-9dd0-f3db1e521186-cert podName:7bc3e182-acec-4866-9dd0-f3db1e521186 nodeName:}" failed. No retries permitted until 2026-07-21 12:46:53.069315452 +0000 UTC m=+426.373320520 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/7bc3e182-acec-4866-9dd0-f3db1e521186-cert") pod "odh-model-controller-7998c67dc4-s7tkd" (UID: "7bc3e182-acec-4866-9dd0-f3db1e521186") : secret "odh-model-controller-webhook-cert" not found
Jul 21 12:46:52.583543 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:46:52.583514    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-btncj\" (UniqueName: \"kubernetes.io/projected/7bc3e182-acec-4866-9dd0-f3db1e521186-kube-api-access-btncj\") pod \"odh-model-controller-7998c67dc4-s7tkd\" (UID: \"7bc3e182-acec-4866-9dd0-f3db1e521186\") " pod="opendatahub/odh-model-controller-7998c67dc4-s7tkd"
Jul 21 12:46:52.617205 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:46:52.617180    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-67cb6d6b5d-rfwlt"
Jul 21 12:46:52.764621 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:46:52.764590    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-67cb6d6b5d-rfwlt"]
Jul 21 12:46:52.768060 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:46:52.768032    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod041ed91e_0571_4917_a641_c554b8734229.slice/crio-80334a96741c0ec78c6e11d307b4142b0fa3ab72b2ba7f607947cb7829724257 WatchSource:0}: Error finding container 80334a96741c0ec78c6e11d307b4142b0fa3ab72b2ba7f607947cb7829724257: Status 404 returned error can't find the container with id 80334a96741c0ec78c6e11d307b4142b0fa3ab72b2ba7f607947cb7829724257
Jul 21 12:46:53.026428 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:46:53.026395    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-67cb6d6b5d-rfwlt" event={"ID":"041ed91e-0571-4917-a641-c554b8734229","Type":"ContainerStarted","Data":"80334a96741c0ec78c6e11d307b4142b0fa3ab72b2ba7f607947cb7829724257"}
Jul 21 12:46:53.073086 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:46:53.073057    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7bc3e182-acec-4866-9dd0-f3db1e521186-cert\") pod \"odh-model-controller-7998c67dc4-s7tkd\" (UID: \"7bc3e182-acec-4866-9dd0-f3db1e521186\") " pod="opendatahub/odh-model-controller-7998c67dc4-s7tkd"
Jul 21 12:46:53.075588 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:46:53.075570    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7bc3e182-acec-4866-9dd0-f3db1e521186-cert\") pod \"odh-model-controller-7998c67dc4-s7tkd\" (UID: \"7bc3e182-acec-4866-9dd0-f3db1e521186\") " pod="opendatahub/odh-model-controller-7998c67dc4-s7tkd"
Jul 21 12:46:53.342803 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:46:53.342723    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-s7tkd"
Jul 21 12:46:53.493703 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:46:53.493668    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-phbht"]
Jul 21 12:46:53.497499 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:46:53.497476    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-phbht"
Jul 21 12:46:53.499959 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:46:53.499937    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Jul 21 12:46:53.508989 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:46:53.508853    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-s7tkd"]
Jul 21 12:46:53.510950 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:46:53.510928    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-phbht"]
Jul 21 12:46:53.512562 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:46:53.512535    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7bc3e182_acec_4866_9dd0_f3db1e521186.slice/crio-509f191444d5af66f81c3dcbddcc0be5fecaaccebb4803358bed09cd8d610881 WatchSource:0}: Error finding container 509f191444d5af66f81c3dcbddcc0be5fecaaccebb4803358bed09cd8d610881: Status 404 returned error can't find the container with id 509f191444d5af66f81c3dcbddcc0be5fecaaccebb4803358bed09cd8d610881
Jul 21 12:46:53.578049 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:46:53.578007    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wm4jt\" (UniqueName: \"kubernetes.io/projected/a9b7bd6c-d5fa-4c79-890e-7509ba41dd4f-kube-api-access-wm4jt\") pod \"model-serving-api-54d89bd5fc-phbht\" (UID: \"a9b7bd6c-d5fa-4c79-890e-7509ba41dd4f\") " pod="opendatahub/model-serving-api-54d89bd5fc-phbht"
Jul 21 12:46:53.578225 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:46:53.578101    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a9b7bd6c-d5fa-4c79-890e-7509ba41dd4f-tls-certs\") pod \"model-serving-api-54d89bd5fc-phbht\" (UID: \"a9b7bd6c-d5fa-4c79-890e-7509ba41dd4f\") " pod="opendatahub/model-serving-api-54d89bd5fc-phbht"
Jul 21 12:46:53.672463 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:46:53.672380    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-57h82"]
Jul 21 12:46:53.676271 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:46:53.676248    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-57h82"
Jul 21 12:46:53.678517 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:46:53.678496    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 21 12:46:53.678517 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:46:53.678507    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 21 12:46:53.678705 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:46:53.678532    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 21 12:46:53.678705 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:46:53.678539    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wm4jt\" (UniqueName: \"kubernetes.io/projected/a9b7bd6c-d5fa-4c79-890e-7509ba41dd4f-kube-api-access-wm4jt\") pod \"model-serving-api-54d89bd5fc-phbht\" (UID: \"a9b7bd6c-d5fa-4c79-890e-7509ba41dd4f\") " pod="opendatahub/model-serving-api-54d89bd5fc-phbht"
Jul 21 12:46:53.678705 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:46:53.678616    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a9b7bd6c-d5fa-4c79-890e-7509ba41dd4f-tls-certs\") pod \"model-serving-api-54d89bd5fc-phbht\" (UID: \"a9b7bd6c-d5fa-4c79-890e-7509ba41dd4f\") " pod="opendatahub/model-serving-api-54d89bd5fc-phbht"
Jul 21 12:46:53.678856 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:46:53.678754    2556 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found
Jul 21 12:46:53.678856 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:46:53.678826    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a9b7bd6c-d5fa-4c79-890e-7509ba41dd4f-tls-certs podName:a9b7bd6c-d5fa-4c79-890e-7509ba41dd4f nodeName:}" failed. No retries permitted until 2026-07-21 12:46:54.178807603 +0000 UTC m=+427.482812688 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/a9b7bd6c-d5fa-4c79-890e-7509ba41dd4f-tls-certs") pod "model-serving-api-54d89bd5fc-phbht" (UID: "a9b7bd6c-d5fa-4c79-890e-7509ba41dd4f") : secret "model-serving-api-tls" not found
Jul 21 12:46:53.678856 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:46:53.678833    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-q58rx\""
Jul 21 12:46:53.678856 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:46:53.678851    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 21 12:46:53.691074 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:46:53.691002    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wm4jt\" (UniqueName: \"kubernetes.io/projected/a9b7bd6c-d5fa-4c79-890e-7509ba41dd4f-kube-api-access-wm4jt\") pod \"model-serving-api-54d89bd5fc-phbht\" (UID: \"a9b7bd6c-d5fa-4c79-890e-7509ba41dd4f\") " pod="opendatahub/model-serving-api-54d89bd5fc-phbht"
Jul 21 12:46:53.692710 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:46:53.692683    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-57h82"]
Jul 21 12:46:53.779087 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:46:53.779054    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/43351ecc-b6e6-418d-8217-a7253f657a54-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-57h82\" (UID: \"43351ecc-b6e6-418d-8217-a7253f657a54\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-57h82"
Jul 21 12:46:53.779271 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:46:53.779096    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/43351ecc-b6e6-418d-8217-a7253f657a54-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-57h82\" (UID: \"43351ecc-b6e6-418d-8217-a7253f657a54\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-57h82"
Jul 21 12:46:53.779271 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:46:53.779119    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/43351ecc-b6e6-418d-8217-a7253f657a54-istio-token\") pod \"istiod-openshift-gateway-7574459656-57h82\" (UID: \"43351ecc-b6e6-418d-8217-a7253f657a54\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-57h82"
Jul 21 12:46:53.779271 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:46:53.779183    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/43351ecc-b6e6-418d-8217-a7253f657a54-cacerts\") pod \"istiod-openshift-gateway-7574459656-57h82\" (UID: \"43351ecc-b6e6-418d-8217-a7253f657a54\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-57h82"
Jul 21 12:46:53.779438 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:46:53.779284    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/43351ecc-b6e6-418d-8217-a7253f657a54-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-57h82\" (UID: \"43351ecc-b6e6-418d-8217-a7253f657a54\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-57h82"
Jul 21 12:46:53.779438 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:46:53.779323    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/43351ecc-b6e6-418d-8217-a7253f657a54-local-certs\") pod \"istiod-openshift-gateway-7574459656-57h82\" (UID: \"43351ecc-b6e6-418d-8217-a7253f657a54\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-57h82"
Jul 21 12:46:53.779438 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:46:53.779347    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jdgnb\" (UniqueName: \"kubernetes.io/projected/43351ecc-b6e6-418d-8217-a7253f657a54-kube-api-access-jdgnb\") pod \"istiod-openshift-gateway-7574459656-57h82\" (UID: \"43351ecc-b6e6-418d-8217-a7253f657a54\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-57h82"
Jul 21 12:46:53.880000 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:46:53.879953    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/43351ecc-b6e6-418d-8217-a7253f657a54-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-57h82\" (UID: \"43351ecc-b6e6-418d-8217-a7253f657a54\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-57h82"
Jul 21 12:46:53.880000 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:46:53.880028    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/43351ecc-b6e6-418d-8217-a7253f657a54-local-certs\") pod \"istiod-openshift-gateway-7574459656-57h82\" (UID: \"43351ecc-b6e6-418d-8217-a7253f657a54\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-57h82"
Jul 21 12:46:53.880228 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:46:53.880060    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jdgnb\" (UniqueName: \"kubernetes.io/projected/43351ecc-b6e6-418d-8217-a7253f657a54-kube-api-access-jdgnb\") pod \"istiod-openshift-gateway-7574459656-57h82\" (UID: \"43351ecc-b6e6-418d-8217-a7253f657a54\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-57h82"
Jul 21 12:46:53.880228 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:46:53.880086    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/43351ecc-b6e6-418d-8217-a7253f657a54-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-57h82\" (UID: \"43351ecc-b6e6-418d-8217-a7253f657a54\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-57h82"
Jul 21 12:46:53.880228 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:46:53.880113    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/43351ecc-b6e6-418d-8217-a7253f657a54-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-57h82\" (UID: \"43351ecc-b6e6-418d-8217-a7253f657a54\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-57h82"
Jul 21 12:46:53.880388 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:46:53.880236    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/43351ecc-b6e6-418d-8217-a7253f657a54-istio-token\") pod \"istiod-openshift-gateway-7574459656-57h82\" (UID: \"43351ecc-b6e6-418d-8217-a7253f657a54\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-57h82"
Jul 21 12:46:53.880388 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:46:53.880297    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/43351ecc-b6e6-418d-8217-a7253f657a54-cacerts\") pod \"istiod-openshift-gateway-7574459656-57h82\" (UID: \"43351ecc-b6e6-418d-8217-a7253f657a54\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-57h82"
Jul 21 12:46:53.880807 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:46:53.880786    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/43351ecc-b6e6-418d-8217-a7253f657a54-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-57h82\" (UID: \"43351ecc-b6e6-418d-8217-a7253f657a54\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-57h82"
Jul 21 12:46:53.882490 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:46:53.882460    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/43351ecc-b6e6-418d-8217-a7253f657a54-local-certs\") pod \"istiod-openshift-gateway-7574459656-57h82\" (UID: \"43351ecc-b6e6-418d-8217-a7253f657a54\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-57h82"
Jul 21 12:46:53.882652 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:46:53.882621    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/43351ecc-b6e6-418d-8217-a7253f657a54-cacerts\") pod \"istiod-openshift-gateway-7574459656-57h82\" (UID: \"43351ecc-b6e6-418d-8217-a7253f657a54\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-57h82"
Jul 21 12:46:53.882855 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:46:53.882837    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/43351ecc-b6e6-418d-8217-a7253f657a54-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-57h82\" (UID: \"43351ecc-b6e6-418d-8217-a7253f657a54\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-57h82"
Jul 21 12:46:53.882947 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:46:53.882926    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/43351ecc-b6e6-418d-8217-a7253f657a54-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-57h82\" (UID: \"43351ecc-b6e6-418d-8217-a7253f657a54\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-57h82"
Jul 21 12:46:53.891519 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:46:53.891489    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/43351ecc-b6e6-418d-8217-a7253f657a54-istio-token\") pod \"istiod-openshift-gateway-7574459656-57h82\" (UID: \"43351ecc-b6e6-418d-8217-a7253f657a54\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-57h82"
Jul 21 12:46:53.891846 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:46:53.891826    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jdgnb\" (UniqueName: \"kubernetes.io/projected/43351ecc-b6e6-418d-8217-a7253f657a54-kube-api-access-jdgnb\") pod \"istiod-openshift-gateway-7574459656-57h82\" (UID: \"43351ecc-b6e6-418d-8217-a7253f657a54\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-57h82"
Jul 21 12:46:53.988895 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:46:53.988865    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-57h82"
Jul 21 12:46:54.033303 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:46:54.033268    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-s7tkd" event={"ID":"7bc3e182-acec-4866-9dd0-f3db1e521186","Type":"ContainerStarted","Data":"509f191444d5af66f81c3dcbddcc0be5fecaaccebb4803358bed09cd8d610881"}
Jul 21 12:46:54.141925 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:46:54.141900    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-57h82"]
Jul 21 12:46:54.144888 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:46:54.144859    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43351ecc_b6e6_418d_8217_a7253f657a54.slice/crio-fb7d052409dd424759cb77021a62d5d1138e0c835c9c98387d083e8f76c8fd58 WatchSource:0}: Error finding container fb7d052409dd424759cb77021a62d5d1138e0c835c9c98387d083e8f76c8fd58: Status 404 returned error can't find the container with id fb7d052409dd424759cb77021a62d5d1138e0c835c9c98387d083e8f76c8fd58
Jul 21 12:46:54.182510 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:46:54.182483    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a9b7bd6c-d5fa-4c79-890e-7509ba41dd4f-tls-certs\") pod \"model-serving-api-54d89bd5fc-phbht\" (UID: \"a9b7bd6c-d5fa-4c79-890e-7509ba41dd4f\") " pod="opendatahub/model-serving-api-54d89bd5fc-phbht"
Jul 21 12:46:54.185295 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:46:54.185264    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a9b7bd6c-d5fa-4c79-890e-7509ba41dd4f-tls-certs\") pod \"model-serving-api-54d89bd5fc-phbht\" (UID: \"a9b7bd6c-d5fa-4c79-890e-7509ba41dd4f\") " pod="opendatahub/model-serving-api-54d89bd5fc-phbht"
Jul 21 12:46:54.411131 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:46:54.411038    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-phbht"
Jul 21 12:46:54.585042 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:46:54.584946    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-phbht"]
Jul 21 12:46:55.045133 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:46:55.045094    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-57h82" event={"ID":"43351ecc-b6e6-418d-8217-a7253f657a54","Type":"ContainerStarted","Data":"fb7d052409dd424759cb77021a62d5d1138e0c835c9c98387d083e8f76c8fd58"}
Jul 21 12:46:55.415315 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:46:55.415238    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9b7bd6c_d5fa_4c79_890e_7509ba41dd4f.slice/crio-3535e0ec9d13a2fb1b64fd79106e2151e7bba369d8c895886cd8ed66a9dbfe58 WatchSource:0}: Error finding container 3535e0ec9d13a2fb1b64fd79106e2151e7bba369d8c895886cd8ed66a9dbfe58: Status 404 returned error can't find the container with id 3535e0ec9d13a2fb1b64fd79106e2151e7bba369d8c895886cd8ed66a9dbfe58
Jul 21 12:46:56.053942 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:46:56.053897    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-phbht" event={"ID":"a9b7bd6c-d5fa-4c79-890e-7509ba41dd4f","Type":"ContainerStarted","Data":"3535e0ec9d13a2fb1b64fd79106e2151e7bba369d8c895886cd8ed66a9dbfe58"}
Jul 21 12:46:57.062881 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:46:57.062078    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-67cb6d6b5d-rfwlt" event={"ID":"041ed91e-0571-4917-a641-c554b8734229","Type":"ContainerStarted","Data":"78453095dbb3aecd567d5f371c34f34518c850be0839230ec3cc7e33df94da00"}
Jul 21 12:46:57.064527 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:46:57.064422    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-s7tkd" event={"ID":"7bc3e182-acec-4866-9dd0-f3db1e521186","Type":"ContainerStarted","Data":"840e6670d2a37e4d53b4d6b4b0d47411cc2764994707b1639c5fbec5bc7b083f"}
Jul 21 12:46:57.064867 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:46:57.064816    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-s7tkd"
Jul 21 12:46:57.082811 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:46:57.082743    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-67cb6d6b5d-rfwlt" podStartSLOduration=1.067086296 podStartE2EDuration="5.082718561s" totalImagesPullingTime="4.015632265s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:46:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:46:57.082124264 +0000 UTC m=+430.386129369" watchObservedRunningTime="2026-07-21 12:46:57.082718561 +0000 UTC m=+430.386723720"
Jul 21 12:46:57.102306 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:46:57.102253    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-s7tkd" podStartSLOduration=1.7846441130000001 podStartE2EDuration="5.102230178s" totalImagesPullingTime="3.317586065s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:46:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:46:57.10015095 +0000 UTC m=+430.404156040" watchObservedRunningTime="2026-07-21 12:46:57.102230178 +0000 UTC m=+430.406235269"
Jul 21 12:46:58.574269 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:46:58.574235    2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 21 12:46:58.574558 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:46:58.574299    2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 21 12:46:59.077545 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:46:59.077501    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-57h82" event={"ID":"43351ecc-b6e6-418d-8217-a7253f657a54","Type":"ContainerStarted","Data":"44632d6ecc4a5af58e74b658bc05fbd83deddbe5c0b72152e9b9c842512738ed"}
Jul 21 12:46:59.077746 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:46:59.077654    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-57h82"
Jul 21 12:46:59.079360 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:46:59.079331    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-phbht" event={"ID":"a9b7bd6c-d5fa-4c79-890e-7509ba41dd4f","Type":"ContainerStarted","Data":"d8c90183968d631df8a93c98bc738bbed7a972afd6923c92d8ab81f0b683123f"}
Jul 21 12:46:59.079476 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:46:59.079384    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-phbht"
Jul 21 12:46:59.079752 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:46:59.079723    2556 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-57h82 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 21 12:46:59.079835 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:46:59.079773    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-57h82" podUID="43351ecc-b6e6-418d-8217-a7253f657a54" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 21 12:46:59.128597 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:46:59.128544    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-57h82" podStartSLOduration=1.701706698 podStartE2EDuration="6.12852771s" totalImagesPullingTime="4.426821012s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:46:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:46:59.124369966 +0000 UTC m=+432.428375074" watchObservedRunningTime="2026-07-21 12:46:59.12852771 +0000 UTC m=+432.432532800"
Jul 21 12:46:59.152802 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:46:59.152753    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-phbht" podStartSLOduration=2.996362262 podStartE2EDuration="6.1527364s" totalImagesPullingTime="3.156374138s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:46:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:46:59.151746296 +0000 UTC m=+432.455751387" watchObservedRunningTime="2026-07-21 12:46:59.1527364 +0000 UTC m=+432.456741579"
Jul 21 12:47:00.026007 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:00.025974    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gldrd"]
Jul 21 12:47:00.029656 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:00.029637    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gldrd"
Jul 21 12:47:00.032431 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:00.032409    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-gv428\""
Jul 21 12:47:00.043458 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:00.043432    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gldrd"]
Jul 21 12:47:00.084525 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:00.084495    2556 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-57h82 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 21 12:47:00.084674 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:00.084545    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-57h82" podUID="43351ecc-b6e6-418d-8217-a7253f657a54" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 21 12:47:00.141975 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:00.141942    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/856c41ff-2032-4fbd-91a2-44a28c8682ce-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4gldrd\" (UID: \"856c41ff-2032-4fbd-91a2-44a28c8682ce\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gldrd"
Jul 21 12:47:00.142166 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:00.141993    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/856c41ff-2032-4fbd-91a2-44a28c8682ce-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4gldrd\" (UID: \"856c41ff-2032-4fbd-91a2-44a28c8682ce\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gldrd"
Jul 21 12:47:00.142166 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:00.142052    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/856c41ff-2032-4fbd-91a2-44a28c8682ce-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4gldrd\" (UID: \"856c41ff-2032-4fbd-91a2-44a28c8682ce\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gldrd"
Jul 21 12:47:00.142166 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:00.142084    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/856c41ff-2032-4fbd-91a2-44a28c8682ce-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4gldrd\" (UID: \"856c41ff-2032-4fbd-91a2-44a28c8682ce\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gldrd"
Jul 21 12:47:00.142166 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:00.142139    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/856c41ff-2032-4fbd-91a2-44a28c8682ce-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4gldrd\" (UID: \"856c41ff-2032-4fbd-91a2-44a28c8682ce\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gldrd"
Jul 21 12:47:00.142334 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:00.142200    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/856c41ff-2032-4fbd-91a2-44a28c8682ce-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4gldrd\" (UID: \"856c41ff-2032-4fbd-91a2-44a28c8682ce\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gldrd"
Jul 21 12:47:00.142334 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:00.142240    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/856c41ff-2032-4fbd-91a2-44a28c8682ce-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4gldrd\" (UID: \"856c41ff-2032-4fbd-91a2-44a28c8682ce\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gldrd"
Jul 21 12:47:00.142334 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:00.142325    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6hjh7\" (UniqueName: \"kubernetes.io/projected/856c41ff-2032-4fbd-91a2-44a28c8682ce-kube-api-access-6hjh7\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4gldrd\" (UID: \"856c41ff-2032-4fbd-91a2-44a28c8682ce\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gldrd"
Jul 21 12:47:00.142445 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:00.142386    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/856c41ff-2032-4fbd-91a2-44a28c8682ce-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4gldrd\" (UID: \"856c41ff-2032-4fbd-91a2-44a28c8682ce\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gldrd"
Jul 21 12:47:00.243158 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:00.243125    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/856c41ff-2032-4fbd-91a2-44a28c8682ce-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4gldrd\" (UID: \"856c41ff-2032-4fbd-91a2-44a28c8682ce\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gldrd"
Jul 21 12:47:00.243158 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:00.243161    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/856c41ff-2032-4fbd-91a2-44a28c8682ce-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4gldrd\" (UID: \"856c41ff-2032-4fbd-91a2-44a28c8682ce\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gldrd"
Jul 21 12:47:00.243409 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:00.243191    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/856c41ff-2032-4fbd-91a2-44a28c8682ce-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4gldrd\" (UID: \"856c41ff-2032-4fbd-91a2-44a28c8682ce\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gldrd"
Jul 21 12:47:00.243409 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:00.243221    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/856c41ff-2032-4fbd-91a2-44a28c8682ce-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4gldrd\" (UID: \"856c41ff-2032-4fbd-91a2-44a28c8682ce\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gldrd"
Jul 21 12:47:00.243409 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:00.243245    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/856c41ff-2032-4fbd-91a2-44a28c8682ce-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4gldrd\" (UID: \"856c41ff-2032-4fbd-91a2-44a28c8682ce\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gldrd"
Jul 21 12:47:00.243409 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:00.243302    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6hjh7\" (UniqueName: \"kubernetes.io/projected/856c41ff-2032-4fbd-91a2-44a28c8682ce-kube-api-access-6hjh7\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4gldrd\" (UID: \"856c41ff-2032-4fbd-91a2-44a28c8682ce\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gldrd"
Jul 21 12:47:00.243409 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:00.243368    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/856c41ff-2032-4fbd-91a2-44a28c8682ce-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4gldrd\" (UID: \"856c41ff-2032-4fbd-91a2-44a28c8682ce\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gldrd"
Jul 21 12:47:00.243659 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:00.243443    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/856c41ff-2032-4fbd-91a2-44a28c8682ce-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4gldrd\" (UID: \"856c41ff-2032-4fbd-91a2-44a28c8682ce\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gldrd"
Jul 21 12:47:00.243659 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:00.243495    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/856c41ff-2032-4fbd-91a2-44a28c8682ce-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4gldrd\" (UID: \"856c41ff-2032-4fbd-91a2-44a28c8682ce\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gldrd"
Jul 21 12:47:00.243659 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:00.243616    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/856c41ff-2032-4fbd-91a2-44a28c8682ce-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4gldrd\" (UID: \"856c41ff-2032-4fbd-91a2-44a28c8682ce\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gldrd"
Jul 21 12:47:00.243816 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:00.243754    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/856c41ff-2032-4fbd-91a2-44a28c8682ce-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4gldrd\" (UID: \"856c41ff-2032-4fbd-91a2-44a28c8682ce\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gldrd"
Jul 21 12:47:00.244037 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:00.243987    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/856c41ff-2032-4fbd-91a2-44a28c8682ce-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4gldrd\" (UID: \"856c41ff-2032-4fbd-91a2-44a28c8682ce\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gldrd"
Jul 21 12:47:00.244171 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:00.244092    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/856c41ff-2032-4fbd-91a2-44a28c8682ce-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4gldrd\" (UID: \"856c41ff-2032-4fbd-91a2-44a28c8682ce\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gldrd"
Jul 21 12:47:00.244374 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:00.244352    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/856c41ff-2032-4fbd-91a2-44a28c8682ce-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4gldrd\" (UID: \"856c41ff-2032-4fbd-91a2-44a28c8682ce\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gldrd"
Jul 21 12:47:00.246350 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:00.246327    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/856c41ff-2032-4fbd-91a2-44a28c8682ce-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4gldrd\" (UID: \"856c41ff-2032-4fbd-91a2-44a28c8682ce\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gldrd"
Jul 21 12:47:00.246464 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:00.246334    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/856c41ff-2032-4fbd-91a2-44a28c8682ce-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4gldrd\" (UID: \"856c41ff-2032-4fbd-91a2-44a28c8682ce\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gldrd"
Jul 21 12:47:00.264486 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:00.264457    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/856c41ff-2032-4fbd-91a2-44a28c8682ce-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4gldrd\" (UID: \"856c41ff-2032-4fbd-91a2-44a28c8682ce\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gldrd"
Jul 21 12:47:00.266541 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:00.266522    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6hjh7\" (UniqueName: \"kubernetes.io/projected/856c41ff-2032-4fbd-91a2-44a28c8682ce-kube-api-access-6hjh7\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4gldrd\" (UID: \"856c41ff-2032-4fbd-91a2-44a28c8682ce\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gldrd"
Jul 21 12:47:00.341361 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:00.341287    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gldrd"
Jul 21 12:47:00.508672 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:00.508644    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gldrd"]
Jul 21 12:47:00.510767 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:47:00.510740    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod856c41ff_2032_4fbd_91a2_44a28c8682ce.slice/crio-be155c318628de494a66367be38b92b13564e16e041c363d74ce12057ac3acff WatchSource:0}: Error finding container be155c318628de494a66367be38b92b13564e16e041c363d74ce12057ac3acff: Status 404 returned error can't find the container with id be155c318628de494a66367be38b92b13564e16e041c363d74ce12057ac3acff
Jul 21 12:47:01.089108 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:01.089068    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gldrd" event={"ID":"856c41ff-2032-4fbd-91a2-44a28c8682ce","Type":"ContainerStarted","Data":"be155c318628de494a66367be38b92b13564e16e041c363d74ce12057ac3acff"}
Jul 21 12:47:02.780033 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:02.779970    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-6r5kh"]
Jul 21 12:47:02.787659 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:02.787630    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-6r5kh"
Jul 21 12:47:02.788309 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:02.788279    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-94vcn"]
Jul 21 12:47:02.790203 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:02.789984    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-khrs2\""
Jul 21 12:47:02.790471 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:02.790444    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Jul 21 12:47:02.792488 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:02.792425    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-94vcn"
Jul 21 12:47:02.795791 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:02.795772    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-6r5kh"]
Jul 21 12:47:02.796567 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:02.795846    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Jul 21 12:47:02.796567 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:02.796145    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-9dsk4\""
Jul 21 12:47:02.803642 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:02.803620    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-94vcn"]
Jul 21 12:47:02.869973 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:02.869935    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xp8jv\" (UniqueName: \"kubernetes.io/projected/68425978-0369-4748-ba6f-b3cac04a2552-kube-api-access-xp8jv\") pod \"kserve-controller-manager-d6f4b65b8-6r5kh\" (UID: \"68425978-0369-4748-ba6f-b3cac04a2552\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-6r5kh"
Jul 21 12:47:02.870191 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:02.869998    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k4wwj\" (UniqueName: \"kubernetes.io/projected/7c34ea1b-966c-490f-9cbb-f9ff00bc95d8-kube-api-access-k4wwj\") pod \"llmisvc-controller-manager-777c6bd7d5-94vcn\" (UID: \"7c34ea1b-966c-490f-9cbb-f9ff00bc95d8\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-94vcn"
Jul 21 12:47:02.870191 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:02.870084    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/68425978-0369-4748-ba6f-b3cac04a2552-cert\") pod \"kserve-controller-manager-d6f4b65b8-6r5kh\" (UID: \"68425978-0369-4748-ba6f-b3cac04a2552\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-6r5kh"
Jul 21 12:47:02.870191 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:02.870170    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7c34ea1b-966c-490f-9cbb-f9ff00bc95d8-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-94vcn\" (UID: \"7c34ea1b-966c-490f-9cbb-f9ff00bc95d8\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-94vcn"
Jul 21 12:47:02.971029 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:02.970974    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k4wwj\" (UniqueName: \"kubernetes.io/projected/7c34ea1b-966c-490f-9cbb-f9ff00bc95d8-kube-api-access-k4wwj\") pod \"llmisvc-controller-manager-777c6bd7d5-94vcn\" (UID: \"7c34ea1b-966c-490f-9cbb-f9ff00bc95d8\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-94vcn"
Jul 21 12:47:02.971206 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:02.971049    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/68425978-0369-4748-ba6f-b3cac04a2552-cert\") pod \"kserve-controller-manager-d6f4b65b8-6r5kh\" (UID: \"68425978-0369-4748-ba6f-b3cac04a2552\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-6r5kh"
Jul 21 12:47:02.971206 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:02.971105    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7c34ea1b-966c-490f-9cbb-f9ff00bc95d8-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-94vcn\" (UID: \"7c34ea1b-966c-490f-9cbb-f9ff00bc95d8\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-94vcn"
Jul 21 12:47:02.971206 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:02.971202    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xp8jv\" (UniqueName: \"kubernetes.io/projected/68425978-0369-4748-ba6f-b3cac04a2552-kube-api-access-xp8jv\") pod \"kserve-controller-manager-d6f4b65b8-6r5kh\" (UID: \"68425978-0369-4748-ba6f-b3cac04a2552\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-6r5kh"
Jul 21 12:47:02.974173 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:02.974141    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/68425978-0369-4748-ba6f-b3cac04a2552-cert\") pod \"kserve-controller-manager-d6f4b65b8-6r5kh\" (UID: \"68425978-0369-4748-ba6f-b3cac04a2552\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-6r5kh"
Jul 21 12:47:02.974281 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:02.974234    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7c34ea1b-966c-490f-9cbb-f9ff00bc95d8-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-94vcn\" (UID: \"7c34ea1b-966c-490f-9cbb-f9ff00bc95d8\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-94vcn"
Jul 21 12:47:02.987757 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:02.987717    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k4wwj\" (UniqueName: \"kubernetes.io/projected/7c34ea1b-966c-490f-9cbb-f9ff00bc95d8-kube-api-access-k4wwj\") pod \"llmisvc-controller-manager-777c6bd7d5-94vcn\" (UID: \"7c34ea1b-966c-490f-9cbb-f9ff00bc95d8\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-94vcn"
Jul 21 12:47:02.987890 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:02.987857    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xp8jv\" (UniqueName: \"kubernetes.io/projected/68425978-0369-4748-ba6f-b3cac04a2552-kube-api-access-xp8jv\") pod \"kserve-controller-manager-d6f4b65b8-6r5kh\" (UID: \"68425978-0369-4748-ba6f-b3cac04a2552\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-6r5kh"
Jul 21 12:47:03.086543 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:03.085757    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-57h82"
Jul 21 12:47:03.104211 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:03.104176    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-6r5kh"
Jul 21 12:47:03.113236 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:03.113208    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-94vcn"
Jul 21 12:47:03.266617 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:03.265782    2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 21 12:47:03.266617 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:03.265889    2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 21 12:47:03.266617 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:03.265935    2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 21 12:47:03.356275 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:03.356248    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-94vcn"]
Jul 21 12:47:03.357154 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:47:03.357128    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod7c34ea1b_966c_490f_9cbb_f9ff00bc95d8.slice/crio-41e5ab37ed12c1ac257f72c0de1483c39c2d540de12a36c8cdaee759c0084963 WatchSource:0}: Error finding container 41e5ab37ed12c1ac257f72c0de1483c39c2d540de12a36c8cdaee759c0084963: Status 404 returned error can't find the container with id 41e5ab37ed12c1ac257f72c0de1483c39c2d540de12a36c8cdaee759c0084963
Jul 21 12:47:03.376143 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:03.376121    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-6r5kh"]
Jul 21 12:47:03.379695 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:47:03.379666    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68425978_0369_4748_ba6f_b3cac04a2552.slice/crio-06756adf036dcbb8dca4b1387cd5ca7294572290ee39918a739c65c5b5a90681 WatchSource:0}: Error finding container 06756adf036dcbb8dca4b1387cd5ca7294572290ee39918a739c65c5b5a90681: Status 404 returned error can't find the container with id 06756adf036dcbb8dca4b1387cd5ca7294572290ee39918a739c65c5b5a90681
Jul 21 12:47:03.993061 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:03.993027    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-57h82"]
Jul 21 12:47:03.993471 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:03.993199    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-57h82" podUID="43351ecc-b6e6-418d-8217-a7253f657a54" containerName="discovery" containerID="cri-o://44632d6ecc4a5af58e74b658bc05fbd83deddbe5c0b72152e9b9c842512738ed" gracePeriod=30
Jul 21 12:47:04.109471 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:04.109432    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gldrd" event={"ID":"856c41ff-2032-4fbd-91a2-44a28c8682ce","Type":"ContainerStarted","Data":"855337f02753a1c60acab38eaaa8214b8a68d11bda326a4faebe6e7ad379bba6"}
Jul 21 12:47:04.110804 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:04.110777    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-6r5kh" event={"ID":"68425978-0369-4748-ba6f-b3cac04a2552","Type":"ContainerStarted","Data":"06756adf036dcbb8dca4b1387cd5ca7294572290ee39918a739c65c5b5a90681"}
Jul 21 12:47:04.113482 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:04.113277    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-94vcn" event={"ID":"7c34ea1b-966c-490f-9cbb-f9ff00bc95d8","Type":"ContainerStarted","Data":"41e5ab37ed12c1ac257f72c0de1483c39c2d540de12a36c8cdaee759c0084963"}
Jul 21 12:47:04.147528 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:04.147416    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gldrd" podStartSLOduration=1.394748146 podStartE2EDuration="4.147395855s" totalImagesPullingTime="2.752647709s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:47:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:47:04.146184212 +0000 UTC m=+437.450189329" watchObservedRunningTime="2026-07-21 12:47:04.147395855 +0000 UTC m=+437.451400945"
Jul 21 12:47:04.277194 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:04.277165    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-57h82"
Jul 21 12:47:04.341915 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:04.341884    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gldrd"
Jul 21 12:47:04.347492 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:04.347458    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gldrd"
Jul 21 12:47:04.387124 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:04.386976    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/43351ecc-b6e6-418d-8217-a7253f657a54-local-certs\") pod \"43351ecc-b6e6-418d-8217-a7253f657a54\" (UID: \"43351ecc-b6e6-418d-8217-a7253f657a54\") "
Jul 21 12:47:04.387124 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:04.387039    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/43351ecc-b6e6-418d-8217-a7253f657a54-cacerts\") pod \"43351ecc-b6e6-418d-8217-a7253f657a54\" (UID: \"43351ecc-b6e6-418d-8217-a7253f657a54\") "
Jul 21 12:47:04.387124 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:04.387078    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/43351ecc-b6e6-418d-8217-a7253f657a54-istio-csr-ca-configmap\") pod \"43351ecc-b6e6-418d-8217-a7253f657a54\" (UID: \"43351ecc-b6e6-418d-8217-a7253f657a54\") "
Jul 21 12:47:04.387124 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:04.387102    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/43351ecc-b6e6-418d-8217-a7253f657a54-istio-token\") pod \"43351ecc-b6e6-418d-8217-a7253f657a54\" (UID: \"43351ecc-b6e6-418d-8217-a7253f657a54\") "
Jul 21 12:47:04.387414 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:04.387155    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/43351ecc-b6e6-418d-8217-a7253f657a54-istio-kubeconfig\") pod \"43351ecc-b6e6-418d-8217-a7253f657a54\" (UID: \"43351ecc-b6e6-418d-8217-a7253f657a54\") "
Jul 21 12:47:04.387414 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:04.387186    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/43351ecc-b6e6-418d-8217-a7253f657a54-istio-csr-dns-cert\") pod \"43351ecc-b6e6-418d-8217-a7253f657a54\" (UID: \"43351ecc-b6e6-418d-8217-a7253f657a54\") "
Jul 21 12:47:04.387414 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:04.387231    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jdgnb\" (UniqueName: \"kubernetes.io/projected/43351ecc-b6e6-418d-8217-a7253f657a54-kube-api-access-jdgnb\") pod \"43351ecc-b6e6-418d-8217-a7253f657a54\" (UID: \"43351ecc-b6e6-418d-8217-a7253f657a54\") "
Jul 21 12:47:04.387947 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:04.387900    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/43351ecc-b6e6-418d-8217-a7253f657a54-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "43351ecc-b6e6-418d-8217-a7253f657a54" (UID: "43351ecc-b6e6-418d-8217-a7253f657a54"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 12:47:04.390117 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:04.390090    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/43351ecc-b6e6-418d-8217-a7253f657a54-local-certs" (OuterVolumeSpecName: "local-certs") pod "43351ecc-b6e6-418d-8217-a7253f657a54" (UID: "43351ecc-b6e6-418d-8217-a7253f657a54"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 12:47:04.390895 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:04.390856    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/43351ecc-b6e6-418d-8217-a7253f657a54-istio-token" (OuterVolumeSpecName: "istio-token") pod "43351ecc-b6e6-418d-8217-a7253f657a54" (UID: "43351ecc-b6e6-418d-8217-a7253f657a54"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 12:47:04.391572 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:04.391546    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/43351ecc-b6e6-418d-8217-a7253f657a54-cacerts" (OuterVolumeSpecName: "cacerts") pod "43351ecc-b6e6-418d-8217-a7253f657a54" (UID: "43351ecc-b6e6-418d-8217-a7253f657a54"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 12:47:04.391670 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:04.391643    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/43351ecc-b6e6-418d-8217-a7253f657a54-kube-api-access-jdgnb" (OuterVolumeSpecName: "kube-api-access-jdgnb") pod "43351ecc-b6e6-418d-8217-a7253f657a54" (UID: "43351ecc-b6e6-418d-8217-a7253f657a54"). InnerVolumeSpecName "kube-api-access-jdgnb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 12:47:04.391771 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:04.391748    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/43351ecc-b6e6-418d-8217-a7253f657a54-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "43351ecc-b6e6-418d-8217-a7253f657a54" (UID: "43351ecc-b6e6-418d-8217-a7253f657a54"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 12:47:04.392477 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:04.392441    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/43351ecc-b6e6-418d-8217-a7253f657a54-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "43351ecc-b6e6-418d-8217-a7253f657a54" (UID: "43351ecc-b6e6-418d-8217-a7253f657a54"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 12:47:04.488203 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:04.488070    2556 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/43351ecc-b6e6-418d-8217-a7253f657a54-istio-token\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Jul 21 12:47:04.488203 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:04.488108    2556 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/43351ecc-b6e6-418d-8217-a7253f657a54-istio-kubeconfig\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Jul 21 12:47:04.488203 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:04.488123    2556 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/43351ecc-b6e6-418d-8217-a7253f657a54-istio-csr-dns-cert\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Jul 21 12:47:04.488203 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:04.488136    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jdgnb\" (UniqueName: \"kubernetes.io/projected/43351ecc-b6e6-418d-8217-a7253f657a54-kube-api-access-jdgnb\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Jul 21 12:47:04.488203 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:04.488152    2556 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/43351ecc-b6e6-418d-8217-a7253f657a54-local-certs\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Jul 21 12:47:04.488203 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:04.488164    2556 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/43351ecc-b6e6-418d-8217-a7253f657a54-cacerts\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Jul 21 12:47:04.488203 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:04.488175    2556 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/43351ecc-b6e6-418d-8217-a7253f657a54-istio-csr-ca-configmap\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Jul 21 12:47:05.120497 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:05.120460    2556 generic.go:356] "Generic (PLEG): container finished" podID="43351ecc-b6e6-418d-8217-a7253f657a54" containerID="44632d6ecc4a5af58e74b658bc05fbd83deddbe5c0b72152e9b9c842512738ed" exitCode=0
Jul 21 12:47:05.120965 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:05.120518    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-57h82"
Jul 21 12:47:05.120965 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:05.120559    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-57h82" event={"ID":"43351ecc-b6e6-418d-8217-a7253f657a54","Type":"ContainerDied","Data":"44632d6ecc4a5af58e74b658bc05fbd83deddbe5c0b72152e9b9c842512738ed"}
Jul 21 12:47:05.120965 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:05.120617    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-57h82" event={"ID":"43351ecc-b6e6-418d-8217-a7253f657a54","Type":"ContainerDied","Data":"fb7d052409dd424759cb77021a62d5d1138e0c835c9c98387d083e8f76c8fd58"}
Jul 21 12:47:05.120965 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:05.120643    2556 scope.go:117] "RemoveContainer" containerID="44632d6ecc4a5af58e74b658bc05fbd83deddbe5c0b72152e9b9c842512738ed"
Jul 21 12:47:05.121248 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:05.121229    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gldrd"
Jul 21 12:47:05.122598 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:05.122580    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4gldrd"
Jul 21 12:47:05.130373 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:05.130355    2556 scope.go:117] "RemoveContainer" containerID="44632d6ecc4a5af58e74b658bc05fbd83deddbe5c0b72152e9b9c842512738ed"
Jul 21 12:47:05.130631 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:47:05.130601    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"44632d6ecc4a5af58e74b658bc05fbd83deddbe5c0b72152e9b9c842512738ed\": container with ID starting with 44632d6ecc4a5af58e74b658bc05fbd83deddbe5c0b72152e9b9c842512738ed not found: ID does not exist" containerID="44632d6ecc4a5af58e74b658bc05fbd83deddbe5c0b72152e9b9c842512738ed"
Jul 21 12:47:05.130691 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:05.130640    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"44632d6ecc4a5af58e74b658bc05fbd83deddbe5c0b72152e9b9c842512738ed"} err="failed to get container status \"44632d6ecc4a5af58e74b658bc05fbd83deddbe5c0b72152e9b9c842512738ed\": rpc error: code = NotFound desc = could not find container \"44632d6ecc4a5af58e74b658bc05fbd83deddbe5c0b72152e9b9c842512738ed\": container with ID starting with 44632d6ecc4a5af58e74b658bc05fbd83deddbe5c0b72152e9b9c842512738ed not found: ID does not exist"
Jul 21 12:47:05.164780 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:05.164741    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-57h82"]
Jul 21 12:47:05.172845 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:05.172816    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-57h82"]
Jul 21 12:47:05.204914 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:05.204880    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="43351ecc-b6e6-418d-8217-a7253f657a54" path="/var/lib/kubelet/pods/43351ecc-b6e6-418d-8217-a7253f657a54/volumes"
Jul 21 12:47:07.133169 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:07.133130    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-94vcn" event={"ID":"7c34ea1b-966c-490f-9cbb-f9ff00bc95d8","Type":"ContainerStarted","Data":"ac4d859deb0a4ced08f785ad8fe698a962c440a3ffd88b173a19d6ac00a999e6"}
Jul 21 12:47:07.133664 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:07.133311    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-94vcn"
Jul 21 12:47:07.134618 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:07.134594    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-6r5kh" event={"ID":"68425978-0369-4748-ba6f-b3cac04a2552","Type":"ContainerStarted","Data":"3978fc17d96d1ff8235d1ce81619e8365c5ead6ca1f556dce9ca956ce6f43fe2"}
Jul 21 12:47:07.157821 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:07.157768    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-94vcn" podStartSLOduration=2.189864329 podStartE2EDuration="5.157754108s" totalImagesPullingTime="2.967889779s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:47:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:47:07.156170188 +0000 UTC m=+440.460175280" watchObservedRunningTime="2026-07-21 12:47:07.157754108 +0000 UTC m=+440.461759197"
Jul 21 12:47:07.177680 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:07.177633    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-6r5kh" podStartSLOduration=2.235149306 podStartE2EDuration="5.177622184s" totalImagesPullingTime="2.942472878s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:47:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:47:07.176370809 +0000 UTC m=+440.480375899" watchObservedRunningTime="2026-07-21 12:47:07.177622184 +0000 UTC m=+440.481627313"
Jul 21 12:47:08.073417 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:08.073340    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-s7tkd"
Jul 21 12:47:08.139890 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:08.139861    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-6r5kh"
Jul 21 12:47:10.088164 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:10.088134    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-phbht"
Jul 21 12:47:38.141922 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:38.141893    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-94vcn"
Jul 21 12:47:39.150216 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:39.150185    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-6r5kh"
Jul 21 12:47:53.370152 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:53.370123    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-7hg94"]
Jul 21 12:47:53.370514 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:53.370459    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="43351ecc-b6e6-418d-8217-a7253f657a54" containerName="discovery"
Jul 21 12:47:53.370514 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:53.370470    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="43351ecc-b6e6-418d-8217-a7253f657a54" containerName="discovery"
Jul 21 12:47:53.370582 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:53.370548    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="43351ecc-b6e6-418d-8217-a7253f657a54" containerName="discovery"
Jul 21 12:47:53.373707 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:53.373689    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-7hg94"
Jul 21 12:47:53.407327 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:53.407299    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 21 12:47:53.407526 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:53.407299    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 21 12:47:53.407526 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:53.407299    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-8tjpt\""
Jul 21 12:47:53.414106 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:53.414074    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-7hg94"]
Jul 21 12:47:53.522769 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:53.522726    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x7vdl\" (UniqueName: \"kubernetes.io/projected/180a49a7-5fec-4ed1-8005-0048290cd776-kube-api-access-x7vdl\") pod \"kuadrant-operator-catalog-7hg94\" (UID: \"180a49a7-5fec-4ed1-8005-0048290cd776\") " pod="kuadrant-system/kuadrant-operator-catalog-7hg94"
Jul 21 12:47:53.624127 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:53.624048    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x7vdl\" (UniqueName: \"kubernetes.io/projected/180a49a7-5fec-4ed1-8005-0048290cd776-kube-api-access-x7vdl\") pod \"kuadrant-operator-catalog-7hg94\" (UID: \"180a49a7-5fec-4ed1-8005-0048290cd776\") " pod="kuadrant-system/kuadrant-operator-catalog-7hg94"
Jul 21 12:47:53.635678 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:53.635648    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x7vdl\" (UniqueName: \"kubernetes.io/projected/180a49a7-5fec-4ed1-8005-0048290cd776-kube-api-access-x7vdl\") pod \"kuadrant-operator-catalog-7hg94\" (UID: \"180a49a7-5fec-4ed1-8005-0048290cd776\") " pod="kuadrant-system/kuadrant-operator-catalog-7hg94"
Jul 21 12:47:53.665639 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:53.665613    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-7hg94"]
Jul 21 12:47:53.665925 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:53.665912    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-7hg94"
Jul 21 12:47:53.802737 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:53.802709    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-7hg94"]
Jul 21 12:47:53.806100 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:47:53.806073    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod180a49a7_5fec_4ed1_8005_0048290cd776.slice/crio-4fb1dc0dea2e71dcdc0ea27be091dd0faaa7881b8b8fba92c27aa1ba6575e104 WatchSource:0}: Error finding container 4fb1dc0dea2e71dcdc0ea27be091dd0faaa7881b8b8fba92c27aa1ba6575e104: Status 404 returned error can't find the container with id 4fb1dc0dea2e71dcdc0ea27be091dd0faaa7881b8b8fba92c27aa1ba6575e104
Jul 21 12:47:53.885741 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:53.885674    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-k5kss"]
Jul 21 12:47:53.888648 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:53.888631    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-k5kss"
Jul 21 12:47:53.909051 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:53.909008    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-k5kss"]
Jul 21 12:47:54.032125 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:54.032096    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8zzxf\" (UniqueName: \"kubernetes.io/projected/b134efa5-61c2-46f9-bc3e-50cc5dfc5b61-kube-api-access-8zzxf\") pod \"kuadrant-operator-catalog-k5kss\" (UID: \"b134efa5-61c2-46f9-bc3e-50cc5dfc5b61\") " pod="kuadrant-system/kuadrant-operator-catalog-k5kss"
Jul 21 12:47:54.133413 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:54.133378    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8zzxf\" (UniqueName: \"kubernetes.io/projected/b134efa5-61c2-46f9-bc3e-50cc5dfc5b61-kube-api-access-8zzxf\") pod \"kuadrant-operator-catalog-k5kss\" (UID: \"b134efa5-61c2-46f9-bc3e-50cc5dfc5b61\") " pod="kuadrant-system/kuadrant-operator-catalog-k5kss"
Jul 21 12:47:54.144830 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:54.144765    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8zzxf\" (UniqueName: \"kubernetes.io/projected/b134efa5-61c2-46f9-bc3e-50cc5dfc5b61-kube-api-access-8zzxf\") pod \"kuadrant-operator-catalog-k5kss\" (UID: \"b134efa5-61c2-46f9-bc3e-50cc5dfc5b61\") " pod="kuadrant-system/kuadrant-operator-catalog-k5kss"
Jul 21 12:47:54.198466 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:54.198442    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-k5kss"
Jul 21 12:47:54.337644 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:54.337619    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-k5kss"]
Jul 21 12:47:54.339940 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:47:54.339900    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb134efa5_61c2_46f9_bc3e_50cc5dfc5b61.slice/crio-55561c9c5b06b54ea969083003b40994096c797b663f270be9c8fa85de578e5c WatchSource:0}: Error finding container 55561c9c5b06b54ea969083003b40994096c797b663f270be9c8fa85de578e5c: Status 404 returned error can't find the container with id 55561c9c5b06b54ea969083003b40994096c797b663f270be9c8fa85de578e5c
Jul 21 12:47:54.351186 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:54.351153    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-7hg94" event={"ID":"180a49a7-5fec-4ed1-8005-0048290cd776","Type":"ContainerStarted","Data":"4fb1dc0dea2e71dcdc0ea27be091dd0faaa7881b8b8fba92c27aa1ba6575e104"}
Jul 21 12:47:54.352282 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:54.352256    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-k5kss" event={"ID":"b134efa5-61c2-46f9-bc3e-50cc5dfc5b61","Type":"ContainerStarted","Data":"55561c9c5b06b54ea969083003b40994096c797b663f270be9c8fa85de578e5c"}
Jul 21 12:47:56.364331 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:56.364285    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-7hg94" event={"ID":"180a49a7-5fec-4ed1-8005-0048290cd776","Type":"ContainerStarted","Data":"49542c58c6656a5810628c67bef3780c9b8adc68541212b2147e76f6a7e42fab"}
Jul 21 12:47:56.364857 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:56.364293    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-7hg94" podUID="180a49a7-5fec-4ed1-8005-0048290cd776" containerName="registry-server" containerID="cri-o://49542c58c6656a5810628c67bef3780c9b8adc68541212b2147e76f6a7e42fab" gracePeriod=2
Jul 21 12:47:56.366160 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:56.366136    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-k5kss" event={"ID":"b134efa5-61c2-46f9-bc3e-50cc5dfc5b61","Type":"ContainerStarted","Data":"15efea9f87e1fd95e7469079c8556e9e26f155002f954a7cbcc3a31ae2745c2c"}
Jul 21 12:47:56.384000 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:56.383957    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-7hg94" podStartSLOduration=1.421573259 podStartE2EDuration="3.383943338s" totalImagesPullingTime="1.962370079s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:47:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:47:56.382075819 +0000 UTC m=+489.686080934" watchObservedRunningTime="2026-07-21 12:47:56.383943338 +0000 UTC m=+489.687948428"
Jul 21 12:47:56.399861 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:56.399820    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-k5kss" podStartSLOduration=1.9709756710000002 podStartE2EDuration="3.399806074s" totalImagesPullingTime="1.428830403s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:47:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:47:56.398976418 +0000 UTC m=+489.702981509" watchObservedRunningTime="2026-07-21 12:47:56.399806074 +0000 UTC m=+489.703811163"
Jul 21 12:47:56.611696 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:56.611670    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-7hg94"
Jul 21 12:47:56.759285 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:56.759256    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x7vdl\" (UniqueName: \"kubernetes.io/projected/180a49a7-5fec-4ed1-8005-0048290cd776-kube-api-access-x7vdl\") pod \"180a49a7-5fec-4ed1-8005-0048290cd776\" (UID: \"180a49a7-5fec-4ed1-8005-0048290cd776\") "
Jul 21 12:47:56.761522 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:56.761499    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/180a49a7-5fec-4ed1-8005-0048290cd776-kube-api-access-x7vdl" (OuterVolumeSpecName: "kube-api-access-x7vdl") pod "180a49a7-5fec-4ed1-8005-0048290cd776" (UID: "180a49a7-5fec-4ed1-8005-0048290cd776"). InnerVolumeSpecName "kube-api-access-x7vdl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 12:47:56.860033 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:56.859991    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x7vdl\" (UniqueName: \"kubernetes.io/projected/180a49a7-5fec-4ed1-8005-0048290cd776-kube-api-access-x7vdl\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Jul 21 12:47:57.372116 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:57.372086    2556 generic.go:356] "Generic (PLEG): container finished" podID="180a49a7-5fec-4ed1-8005-0048290cd776" containerID="49542c58c6656a5810628c67bef3780c9b8adc68541212b2147e76f6a7e42fab" exitCode=0
Jul 21 12:47:57.372559 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:57.372128    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-7hg94"
Jul 21 12:47:57.372559 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:57.372173    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-7hg94" event={"ID":"180a49a7-5fec-4ed1-8005-0048290cd776","Type":"ContainerDied","Data":"49542c58c6656a5810628c67bef3780c9b8adc68541212b2147e76f6a7e42fab"}
Jul 21 12:47:57.372559 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:57.372221    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-7hg94" event={"ID":"180a49a7-5fec-4ed1-8005-0048290cd776","Type":"ContainerDied","Data":"4fb1dc0dea2e71dcdc0ea27be091dd0faaa7881b8b8fba92c27aa1ba6575e104"}
Jul 21 12:47:57.372559 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:57.372243    2556 scope.go:117] "RemoveContainer" containerID="49542c58c6656a5810628c67bef3780c9b8adc68541212b2147e76f6a7e42fab"
Jul 21 12:47:57.381185 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:57.381158    2556 scope.go:117] "RemoveContainer" containerID="49542c58c6656a5810628c67bef3780c9b8adc68541212b2147e76f6a7e42fab"
Jul 21 12:47:57.381423 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:47:57.381407    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"49542c58c6656a5810628c67bef3780c9b8adc68541212b2147e76f6a7e42fab\": container with ID starting with 49542c58c6656a5810628c67bef3780c9b8adc68541212b2147e76f6a7e42fab not found: ID does not exist" containerID="49542c58c6656a5810628c67bef3780c9b8adc68541212b2147e76f6a7e42fab"
Jul 21 12:47:57.381477 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:57.381433    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"49542c58c6656a5810628c67bef3780c9b8adc68541212b2147e76f6a7e42fab"} err="failed to get container status \"49542c58c6656a5810628c67bef3780c9b8adc68541212b2147e76f6a7e42fab\": rpc error: code = NotFound desc = could not find container \"49542c58c6656a5810628c67bef3780c9b8adc68541212b2147e76f6a7e42fab\": container with ID starting with 49542c58c6656a5810628c67bef3780c9b8adc68541212b2147e76f6a7e42fab not found: ID does not exist"
Jul 21 12:47:57.391395 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:57.391374    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-7hg94"]
Jul 21 12:47:57.398341 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:57.398321    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-7hg94"]
Jul 21 12:47:59.194727 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:47:59.194678    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="180a49a7-5fec-4ed1-8005-0048290cd776" path="/var/lib/kubelet/pods/180a49a7-5fec-4ed1-8005-0048290cd776/volumes"
Jul 21 12:48:04.198985 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:48:04.198950    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-k5kss"
Jul 21 12:48:04.199423 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:48:04.199007    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-k5kss"
Jul 21 12:48:04.223301 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:48:04.223265    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-k5kss"
Jul 21 12:48:04.432566 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:48:04.432535    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-k5kss"
Jul 21 12:48:21.344778 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:48:21.344746    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-fbbbc87c4-mgm6t"]
Jul 21 12:48:21.345271 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:48:21.345148    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="180a49a7-5fec-4ed1-8005-0048290cd776" containerName="registry-server"
Jul 21 12:48:21.345271 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:48:21.345162    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="180a49a7-5fec-4ed1-8005-0048290cd776" containerName="registry-server"
Jul 21 12:48:21.345271 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:48:21.345228    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="180a49a7-5fec-4ed1-8005-0048290cd776" containerName="registry-server"
Jul 21 12:48:21.348443 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:48:21.348417    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-fbbbc87c4-mgm6t"
Jul 21 12:48:21.351922 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:48:21.351899    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 21 12:48:21.352188 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:48:21.352165    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 21 12:48:21.352631 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:48:21.352615    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 21 12:48:21.353097 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:48:21.353076    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-9m7pl\""
Jul 21 12:48:21.353555 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:48:21.353483    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 21 12:48:21.353555 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:48:21.353543    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 21 12:48:21.353806 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:48:21.353685    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 21 12:48:21.353806 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:48:21.353763    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 21 12:48:21.358517 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:48:21.358500    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 21 12:48:21.365061 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:48:21.365041    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-fbbbc87c4-mgm6t"]
Jul 21 12:48:21.378965 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:48:21.378941    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d3b4d45b-90db-4272-9d7b-995c91fa468e-console-oauth-config\") pod \"console-fbbbc87c4-mgm6t\" (UID: \"d3b4d45b-90db-4272-9d7b-995c91fa468e\") " pod="openshift-console/console-fbbbc87c4-mgm6t"
Jul 21 12:48:21.379083 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:48:21.378981    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d3b4d45b-90db-4272-9d7b-995c91fa468e-console-serving-cert\") pod \"console-fbbbc87c4-mgm6t\" (UID: \"d3b4d45b-90db-4272-9d7b-995c91fa468e\") " pod="openshift-console/console-fbbbc87c4-mgm6t"
Jul 21 12:48:21.379083 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:48:21.378998    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d3b4d45b-90db-4272-9d7b-995c91fa468e-console-config\") pod \"console-fbbbc87c4-mgm6t\" (UID: \"d3b4d45b-90db-4272-9d7b-995c91fa468e\") " pod="openshift-console/console-fbbbc87c4-mgm6t"
Jul 21 12:48:21.379083 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:48:21.379032    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d3b4d45b-90db-4272-9d7b-995c91fa468e-oauth-serving-cert\") pod \"console-fbbbc87c4-mgm6t\" (UID: \"d3b4d45b-90db-4272-9d7b-995c91fa468e\") " pod="openshift-console/console-fbbbc87c4-mgm6t"
Jul 21 12:48:21.379189 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:48:21.379084    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6h9kf\" (UniqueName: \"kubernetes.io/projected/d3b4d45b-90db-4272-9d7b-995c91fa468e-kube-api-access-6h9kf\") pod \"console-fbbbc87c4-mgm6t\" (UID: \"d3b4d45b-90db-4272-9d7b-995c91fa468e\") " pod="openshift-console/console-fbbbc87c4-mgm6t"
Jul 21 12:48:21.379189 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:48:21.379123    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d3b4d45b-90db-4272-9d7b-995c91fa468e-service-ca\") pod \"console-fbbbc87c4-mgm6t\" (UID: \"d3b4d45b-90db-4272-9d7b-995c91fa468e\") " pod="openshift-console/console-fbbbc87c4-mgm6t"
Jul 21 12:48:21.379189 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:48:21.379145    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d3b4d45b-90db-4272-9d7b-995c91fa468e-trusted-ca-bundle\") pod \"console-fbbbc87c4-mgm6t\" (UID: \"d3b4d45b-90db-4272-9d7b-995c91fa468e\") " pod="openshift-console/console-fbbbc87c4-mgm6t"
Jul 21 12:48:21.479852 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:48:21.479821    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d3b4d45b-90db-4272-9d7b-995c91fa468e-service-ca\") pod \"console-fbbbc87c4-mgm6t\" (UID: \"d3b4d45b-90db-4272-9d7b-995c91fa468e\") " pod="openshift-console/console-fbbbc87c4-mgm6t"
Jul 21 12:48:21.480072 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:48:21.479862    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d3b4d45b-90db-4272-9d7b-995c91fa468e-trusted-ca-bundle\") pod \"console-fbbbc87c4-mgm6t\" (UID: \"d3b4d45b-90db-4272-9d7b-995c91fa468e\") " pod="openshift-console/console-fbbbc87c4-mgm6t"
Jul 21 12:48:21.480072 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:48:21.479936    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d3b4d45b-90db-4272-9d7b-995c91fa468e-console-oauth-config\") pod \"console-fbbbc87c4-mgm6t\" (UID: \"d3b4d45b-90db-4272-9d7b-995c91fa468e\") " pod="openshift-console/console-fbbbc87c4-mgm6t"
Jul 21 12:48:21.480072 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:48:21.479965    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d3b4d45b-90db-4272-9d7b-995c91fa468e-console-serving-cert\") pod \"console-fbbbc87c4-mgm6t\" (UID: \"d3b4d45b-90db-4272-9d7b-995c91fa468e\") " pod="openshift-console/console-fbbbc87c4-mgm6t"
Jul 21 12:48:21.480072 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:48:21.479982    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d3b4d45b-90db-4272-9d7b-995c91fa468e-console-config\") pod \"console-fbbbc87c4-mgm6t\" (UID: \"d3b4d45b-90db-4272-9d7b-995c91fa468e\") " pod="openshift-console/console-fbbbc87c4-mgm6t"
Jul 21 12:48:21.480072 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:48:21.480003    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d3b4d45b-90db-4272-9d7b-995c91fa468e-oauth-serving-cert\") pod \"console-fbbbc87c4-mgm6t\" (UID: \"d3b4d45b-90db-4272-9d7b-995c91fa468e\") " pod="openshift-console/console-fbbbc87c4-mgm6t"
Jul 21 12:48:21.480072 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:48:21.480060    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6h9kf\" (UniqueName: \"kubernetes.io/projected/d3b4d45b-90db-4272-9d7b-995c91fa468e-kube-api-access-6h9kf\") pod \"console-fbbbc87c4-mgm6t\" (UID: \"d3b4d45b-90db-4272-9d7b-995c91fa468e\") " pod="openshift-console/console-fbbbc87c4-mgm6t"
Jul 21 12:48:21.480702 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:48:21.480675    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d3b4d45b-90db-4272-9d7b-995c91fa468e-service-ca\") pod \"console-fbbbc87c4-mgm6t\" (UID: \"d3b4d45b-90db-4272-9d7b-995c91fa468e\") " pod="openshift-console/console-fbbbc87c4-mgm6t"
Jul 21 12:48:21.480921 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:48:21.480883    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d3b4d45b-90db-4272-9d7b-995c91fa468e-oauth-serving-cert\") pod \"console-fbbbc87c4-mgm6t\" (UID: \"d3b4d45b-90db-4272-9d7b-995c91fa468e\") " pod="openshift-console/console-fbbbc87c4-mgm6t"
Jul 21 12:48:21.481052 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:48:21.480956    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d3b4d45b-90db-4272-9d7b-995c91fa468e-trusted-ca-bundle\") pod \"console-fbbbc87c4-mgm6t\" (UID: \"d3b4d45b-90db-4272-9d7b-995c91fa468e\") " pod="openshift-console/console-fbbbc87c4-mgm6t"
Jul 21 12:48:21.481238 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:48:21.481218    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d3b4d45b-90db-4272-9d7b-995c91fa468e-console-config\") pod \"console-fbbbc87c4-mgm6t\" (UID: \"d3b4d45b-90db-4272-9d7b-995c91fa468e\") " pod="openshift-console/console-fbbbc87c4-mgm6t"
Jul 21 12:48:21.483432 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:48:21.483397    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d3b4d45b-90db-4272-9d7b-995c91fa468e-console-oauth-config\") pod \"console-fbbbc87c4-mgm6t\" (UID: \"d3b4d45b-90db-4272-9d7b-995c91fa468e\") " pod="openshift-console/console-fbbbc87c4-mgm6t"
Jul 21 12:48:21.483771 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:48:21.483747    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d3b4d45b-90db-4272-9d7b-995c91fa468e-console-serving-cert\") pod \"console-fbbbc87c4-mgm6t\" (UID: \"d3b4d45b-90db-4272-9d7b-995c91fa468e\") " pod="openshift-console/console-fbbbc87c4-mgm6t"
Jul 21 12:48:21.511637 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:48:21.511605    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6h9kf\" (UniqueName: \"kubernetes.io/projected/d3b4d45b-90db-4272-9d7b-995c91fa468e-kube-api-access-6h9kf\") pod \"console-fbbbc87c4-mgm6t\" (UID: \"d3b4d45b-90db-4272-9d7b-995c91fa468e\") " pod="openshift-console/console-fbbbc87c4-mgm6t"
Jul 21 12:48:21.659532 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:48:21.659447    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-fbbbc87c4-mgm6t"
Jul 21 12:48:21.807166 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:48:21.807118    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-fbbbc87c4-mgm6t"]
Jul 21 12:48:21.810399 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:48:21.810348    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3b4d45b_90db_4272_9d7b_995c91fa468e.slice/crio-f00275cfa25cd03b5ad9f2ea84a14d6bf95c2bd54319a44dfc7fd969f47f6414 WatchSource:0}: Error finding container f00275cfa25cd03b5ad9f2ea84a14d6bf95c2bd54319a44dfc7fd969f47f6414: Status 404 returned error can't find the container with id f00275cfa25cd03b5ad9f2ea84a14d6bf95c2bd54319a44dfc7fd969f47f6414
Jul 21 12:48:22.501565 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:48:22.501516    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-fbbbc87c4-mgm6t" event={"ID":"d3b4d45b-90db-4272-9d7b-995c91fa468e","Type":"ContainerStarted","Data":"0c7abf9c4159ab3a9ceafa2ec044a17b77866a84fe664b3768dc15730f943f2c"}
Jul 21 12:48:22.501976 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:48:22.501606    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-fbbbc87c4-mgm6t" event={"ID":"d3b4d45b-90db-4272-9d7b-995c91fa468e","Type":"ContainerStarted","Data":"f00275cfa25cd03b5ad9f2ea84a14d6bf95c2bd54319a44dfc7fd969f47f6414"}
Jul 21 12:48:22.554789 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:48:22.554731    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-fbbbc87c4-mgm6t" podStartSLOduration=1.55471512 podStartE2EDuration="1.55471512s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:48:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:48:22.553499315 +0000 UTC m=+515.857504427" watchObservedRunningTime="2026-07-21 12:48:22.55471512 +0000 UTC m=+515.858720249"
Jul 21 12:48:26.867868 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:48:26.867834    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mqms8"]
Jul 21 12:48:26.871501 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:48:26.871483    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mqms8"
Jul 21 12:48:26.874570 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:48:26.874551    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-622jg\""
Jul 21 12:48:26.893447 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:48:26.893417    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mqms8"]
Jul 21 12:48:26.929662 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:48:26.929631    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r94ts\" (UniqueName: \"kubernetes.io/projected/afd3056a-3199-4492-abf6-8aa8afb611cb-kube-api-access-r94ts\") pod \"kuadrant-operator-controller-manager-8684c9ff49-mqms8\" (UID: \"afd3056a-3199-4492-abf6-8aa8afb611cb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mqms8"
Jul 21 12:48:26.929815 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:48:26.929670    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/afd3056a-3199-4492-abf6-8aa8afb611cb-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-mqms8\" (UID: \"afd3056a-3199-4492-abf6-8aa8afb611cb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mqms8"
Jul 21 12:48:27.030564 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:48:27.030525    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r94ts\" (UniqueName: \"kubernetes.io/projected/afd3056a-3199-4492-abf6-8aa8afb611cb-kube-api-access-r94ts\") pod \"kuadrant-operator-controller-manager-8684c9ff49-mqms8\" (UID: \"afd3056a-3199-4492-abf6-8aa8afb611cb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mqms8"
Jul 21 12:48:27.030740 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:48:27.030589    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/afd3056a-3199-4492-abf6-8aa8afb611cb-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-mqms8\" (UID: \"afd3056a-3199-4492-abf6-8aa8afb611cb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mqms8"
Jul 21 12:48:27.031034 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:48:27.030988    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/afd3056a-3199-4492-abf6-8aa8afb611cb-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-mqms8\" (UID: \"afd3056a-3199-4492-abf6-8aa8afb611cb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mqms8"
Jul 21 12:48:27.045585 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:48:27.045559    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r94ts\" (UniqueName: \"kubernetes.io/projected/afd3056a-3199-4492-abf6-8aa8afb611cb-kube-api-access-r94ts\") pod \"kuadrant-operator-controller-manager-8684c9ff49-mqms8\" (UID: \"afd3056a-3199-4492-abf6-8aa8afb611cb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mqms8"
Jul 21 12:48:27.182420 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:48:27.182329    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mqms8"
Jul 21 12:48:27.355615 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:48:27.355580    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mqms8"]
Jul 21 12:48:27.362504 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:48:27.362474    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podafd3056a_3199_4492_abf6_8aa8afb611cb.slice/crio-cce3069cd76593f23efada98d59950620a233c73886ec8023d1217f358823cb2 WatchSource:0}: Error finding container cce3069cd76593f23efada98d59950620a233c73886ec8023d1217f358823cb2: Status 404 returned error can't find the container with id cce3069cd76593f23efada98d59950620a233c73886ec8023d1217f358823cb2
Jul 21 12:48:27.526876 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:48:27.526842    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mqms8" event={"ID":"afd3056a-3199-4492-abf6-8aa8afb611cb","Type":"ContainerStarted","Data":"cce3069cd76593f23efada98d59950620a233c73886ec8023d1217f358823cb2"}
Jul 21 12:48:28.813865 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:48:28.813821    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9wtcz"]
Jul 21 12:48:28.821905 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:48:28.821876    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9wtcz"
Jul 21 12:48:28.826911 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:48:28.826884    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-cvfth\""
Jul 21 12:48:28.836540 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:48:28.836520    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9wtcz"]
Jul 21 12:48:28.947380 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:48:28.947338    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gxxlb\" (UniqueName: \"kubernetes.io/projected/9be9eed0-62ad-4462-b48a-3629d9c15bd0-kube-api-access-gxxlb\") pod \"limitador-operator-controller-manager-859d7b78bd-9wtcz\" (UID: \"9be9eed0-62ad-4462-b48a-3629d9c15bd0\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9wtcz"
Jul 21 12:48:29.048594 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:48:29.048548    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gxxlb\" (UniqueName: \"kubernetes.io/projected/9be9eed0-62ad-4462-b48a-3629d9c15bd0-kube-api-access-gxxlb\") pod \"limitador-operator-controller-manager-859d7b78bd-9wtcz\" (UID: \"9be9eed0-62ad-4462-b48a-3629d9c15bd0\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9wtcz"
Jul 21 12:48:29.079242 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:48:29.079167    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gxxlb\" (UniqueName: \"kubernetes.io/projected/9be9eed0-62ad-4462-b48a-3629d9c15bd0-kube-api-access-gxxlb\") pod \"limitador-operator-controller-manager-859d7b78bd-9wtcz\" (UID: \"9be9eed0-62ad-4462-b48a-3629d9c15bd0\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9wtcz"
Jul 21 12:48:29.134775 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:48:29.134735    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9wtcz"
Jul 21 12:48:29.309911 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:48:29.309878    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9wtcz"]
Jul 21 12:48:29.312462 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:48:29.312433    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9be9eed0_62ad_4462_b48a_3629d9c15bd0.slice/crio-04d56655ab8331819f1e490501ba5ab34d3551987db522bd8e660ac578e00a1c WatchSource:0}: Error finding container 04d56655ab8331819f1e490501ba5ab34d3551987db522bd8e660ac578e00a1c: Status 404 returned error can't find the container with id 04d56655ab8331819f1e490501ba5ab34d3551987db522bd8e660ac578e00a1c
Jul 21 12:48:29.539588 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:48:29.539551    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9wtcz" event={"ID":"9be9eed0-62ad-4462-b48a-3629d9c15bd0","Type":"ContainerStarted","Data":"04d56655ab8331819f1e490501ba5ab34d3551987db522bd8e660ac578e00a1c"}
Jul 21 12:48:31.188101 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:48:31.187994    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-pmr2h"]
Jul 21 12:48:31.191836 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:48:31.191793    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-pmr2h"
Jul 21 12:48:31.202130 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:48:31.202092    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-gbbwc\""
Jul 21 12:48:31.205464 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:48:31.205441    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-pmr2h"]
Jul 21 12:48:31.271448 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:48:31.271406    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t796q\" (UniqueName: \"kubernetes.io/projected/7962da2c-145f-499e-82d3-be635b223b41-kube-api-access-t796q\") pod \"authorino-operator-75cc8d5cb7-pmr2h\" (UID: \"7962da2c-145f-499e-82d3-be635b223b41\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-pmr2h"
Jul 21 12:48:31.372028 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:48:31.371970    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t796q\" (UniqueName: \"kubernetes.io/projected/7962da2c-145f-499e-82d3-be635b223b41-kube-api-access-t796q\") pod \"authorino-operator-75cc8d5cb7-pmr2h\" (UID: \"7962da2c-145f-499e-82d3-be635b223b41\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-pmr2h"
Jul 21 12:48:31.391096 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:48:31.391056    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t796q\" (UniqueName: \"kubernetes.io/projected/7962da2c-145f-499e-82d3-be635b223b41-kube-api-access-t796q\") pod \"authorino-operator-75cc8d5cb7-pmr2h\" (UID: \"7962da2c-145f-499e-82d3-be635b223b41\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-pmr2h"
Jul 21 12:48:31.507517 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:48:31.507478    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-pmr2h"
Jul 21 12:48:31.553639 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:48:31.553604    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9wtcz" event={"ID":"9be9eed0-62ad-4462-b48a-3629d9c15bd0","Type":"ContainerStarted","Data":"b391f3a1256bab4462b0042efe6d9a42ac463a194cdcdf0bb30622d753b2df40"}
Jul 21 12:48:31.553798 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:48:31.553685    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9wtcz"
Jul 21 12:48:31.607544 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:48:31.607479    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9wtcz" podStartSLOduration=2.027537487 podStartE2EDuration="3.607458054s" totalImagesPullingTime="1.579920567s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:48:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:48:31.587169282 +0000 UTC m=+524.891174374" watchObservedRunningTime="2026-07-21 12:48:31.607458054 +0000 UTC m=+524.911463145"
Jul 21 12:48:31.659936 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:48:31.659899    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-fbbbc87c4-mgm6t"
Jul 21 12:48:31.660140 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:48:31.659964    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-fbbbc87c4-mgm6t"
Jul 21 12:48:31.665523 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:48:31.665501    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-fbbbc87c4-mgm6t"
Jul 21 12:48:32.321797 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:48:32.321722    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-pmr2h"]
Jul 21 12:48:32.323928 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:48:32.323902    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7962da2c_145f_499e_82d3_be635b223b41.slice/crio-dbe4d705028a9eb5a7772ba18fec8ead2a8584102d2d513a1a7a448759f7458e WatchSource:0}: Error finding container dbe4d705028a9eb5a7772ba18fec8ead2a8584102d2d513a1a7a448759f7458e: Status 404 returned error can't find the container with id dbe4d705028a9eb5a7772ba18fec8ead2a8584102d2d513a1a7a448759f7458e
Jul 21 12:48:32.563714 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:48:32.563672    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mqms8" event={"ID":"afd3056a-3199-4492-abf6-8aa8afb611cb","Type":"ContainerStarted","Data":"c39edb1df27fcead1233f510d9365ccd95f602eec481f9926705cdbda922540e"}
Jul 21 12:48:32.563895 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:48:32.563789    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mqms8"
Jul 21 12:48:32.565361 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:48:32.565331    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-pmr2h" event={"ID":"7962da2c-145f-499e-82d3-be635b223b41","Type":"ContainerStarted","Data":"dbe4d705028a9eb5a7772ba18fec8ead2a8584102d2d513a1a7a448759f7458e"}
Jul 21 12:48:32.569192 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:48:32.569170    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-fbbbc87c4-mgm6t"
Jul 21 12:48:32.590931 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:48:32.590887    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mqms8" podStartSLOduration=1.7157769269999998 podStartE2EDuration="6.590874285s" totalImagesPullingTime="4.875097358s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:48:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:48:32.58937661 +0000 UTC m=+525.893381715" watchObservedRunningTime="2026-07-21 12:48:32.590874285 +0000 UTC m=+525.894879376"
Jul 21 12:48:35.585252 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:48:35.585219    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-pmr2h" event={"ID":"7962da2c-145f-499e-82d3-be635b223b41","Type":"ContainerStarted","Data":"b7786a303496b2b6f573e332e42314046b54bd520cf48181649e6050659363f5"}
Jul 21 12:48:35.585656 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:48:35.585474    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-pmr2h"
Jul 21 12:48:35.609481 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:48:35.609437    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-pmr2h" podStartSLOduration=2.233136825 podStartE2EDuration="4.609421311s" totalImagesPullingTime="2.376284486s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:48:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:48:35.608649665 +0000 UTC m=+528.912654756" watchObservedRunningTime="2026-07-21 12:48:35.609421311 +0000 UTC m=+528.913426410"
Jul 21 12:48:42.567799 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:48:42.567711    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9wtcz"
Jul 21 12:48:43.574917 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:48:43.574886    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mqms8"
Jul 21 12:48:45.799589 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:48:45.799558    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mqms8"]
Jul 21 12:48:45.800069 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:48:45.799741    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mqms8" podUID="afd3056a-3199-4492-abf6-8aa8afb611cb" containerName="manager" containerID="cri-o://c39edb1df27fcead1233f510d9365ccd95f602eec481f9926705cdbda922540e" gracePeriod=2
Jul 21 12:48:45.814731 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:48:45.814699    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mqms8"]
Jul 21 12:48:45.818719 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:48:45.818687    2556 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"afd3056a-3199-4492-abf6-8aa8afb611cb\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-8684c9ff49-mqms8\": pods \"kuadrant-operator-controller-manager-8684c9ff49-mqms8\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mqms8"
Jul 21 12:48:45.846538 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:48:45.846511    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9wtcz"]
Jul 21 12:48:45.846708 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:48:45.846689    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9wtcz" podUID="9be9eed0-62ad-4462-b48a-3629d9c15bd0" containerName="manager" containerID="cri-o://b391f3a1256bab4462b0042efe6d9a42ac463a194cdcdf0bb30622d753b2df40" gracePeriod=2
Jul 21 12:48:45.849983 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:48:45.849946    2556 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-mqms8\" is forbidden: User \"system:node:ip-10-0-130-2.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-2.ec2.internal' and this object" podUID="afd3056a-3199-4492-abf6-8aa8afb611cb" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mqms8"
Jul 21 12:48:45.859617 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:48:45.859588    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9wtcz"]
Jul 21 12:48:45.861398 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:48:45.861371    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wcvml"]
Jul 21 12:48:45.861826 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:48:45.861805    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="afd3056a-3199-4492-abf6-8aa8afb611cb" containerName="manager"
Jul 21 12:48:45.861826 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:48:45.861827    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="afd3056a-3199-4492-abf6-8aa8afb611cb" containerName="manager"
Jul 21 12:48:45.862041 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:48:45.861872    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9be9eed0-62ad-4462-b48a-3629d9c15bd0" containerName="manager"
Jul 21 12:48:45.862041 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:48:45.861881    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="9be9eed0-62ad-4462-b48a-3629d9c15bd0" containerName="manager"
Jul 21 12:48:45.862041 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:48:45.861977    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="afd3056a-3199-4492-abf6-8aa8afb611cb" containerName="manager"
Jul 21 12:48:45.862041 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:48:45.861987    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="9be9eed0-62ad-4462-b48a-3629d9c15bd0" containerName="manager"
Jul 21 12:48:45.865039 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:48:45.865006    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wcvml"
Jul 21 12:48:45.871102 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:48:45.871077    2556 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-mqms8\" is forbidden: User \"system:node:ip-10-0-130-2.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-2.ec2.internal' and this object" podUID="afd3056a-3199-4492-abf6-8aa8afb611cb" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mqms8"
Jul 21 12:48:45.874925 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:48:45.874903    2556 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-9wtcz\" is forbidden: User \"system:node:ip-10-0-130-2.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-2.ec2.internal' and this object" podUID="9be9eed0-62ad-4462-b48a-3629d9c15bd0" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9wtcz"
Jul 21 12:48:45.890953 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:48:45.890924    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xj2dd"]
Jul 21 12:48:45.894781 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:48:45.894757    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wcvml"]
Jul 21 12:48:45.894936 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:48:45.894894    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xj2dd"
Jul 21 12:48:45.910293 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:48:45.910263    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xj2dd"]
Jul 21 12:48:45.962777 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:48:45.962729    2556 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-mqms8\" is forbidden: User \"system:node:ip-10-0-130-2.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-2.ec2.internal' and this object" podUID="afd3056a-3199-4492-abf6-8aa8afb611cb" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mqms8"
Jul 21 12:48:45.965364 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:48:45.965260    2556 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-9wtcz\" is forbidden: User \"system:node:ip-10-0-130-2.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-2.ec2.internal' and this object" podUID="9be9eed0-62ad-4462-b48a-3629d9c15bd0" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9wtcz"
Jul 21 12:48:46.013041 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:48:46.012928    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/560dca78-8e2e-4eb5-a8fe-1915d354e0c5-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-wcvml\" (UID: \"560dca78-8e2e-4eb5-a8fe-1915d354e0c5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wcvml"
Jul 21 12:48:46.013197 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:48:46.013086    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zcrcn\" (UniqueName: \"kubernetes.io/projected/560dca78-8e2e-4eb5-a8fe-1915d354e0c5-kube-api-access-zcrcn\") pod \"kuadrant-operator-controller-manager-8684c9ff49-wcvml\" (UID: \"560dca78-8e2e-4eb5-a8fe-1915d354e0c5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wcvml"
Jul 21 12:48:46.013197 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:48:46.013138    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f65f5\" (UniqueName: \"kubernetes.io/projected/ad723ec0-e63c-4a5e-acda-719533071ba2-kube-api-access-f65f5\") pod \"limitador-operator-controller-manager-859d7b78bd-xj2dd\" (UID: \"ad723ec0-e63c-4a5e-acda-719533071ba2\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xj2dd"
Jul 21 12:48:46.094497 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:48:46.094477    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mqms8"
Jul 21 12:48:46.096620 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:48:46.096592    2556 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-mqms8\" is forbidden: User \"system:node:ip-10-0-130-2.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-2.ec2.internal' and this object" podUID="afd3056a-3199-4492-abf6-8aa8afb611cb" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mqms8"
Jul 21 12:48:46.097431 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:48:46.097415    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9wtcz"
Jul 21 12:48:46.098433 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:48:46.098414    2556 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-9wtcz\" is forbidden: User \"system:node:ip-10-0-130-2.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-2.ec2.internal' and this object" podUID="9be9eed0-62ad-4462-b48a-3629d9c15bd0" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9wtcz"
Jul 21 12:48:46.099983 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:48:46.099954    2556 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-mqms8\" is forbidden: User \"system:node:ip-10-0-130-2.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-2.ec2.internal' and this object" podUID="afd3056a-3199-4492-abf6-8aa8afb611cb" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mqms8"
Jul 21 12:48:46.101816 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:48:46.101797    2556 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-9wtcz\" is forbidden: User \"system:node:ip-10-0-130-2.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-2.ec2.internal' and this object" podUID="9be9eed0-62ad-4462-b48a-3629d9c15bd0" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9wtcz"
Jul 21 12:48:46.113831 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:48:46.113810    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f65f5\" (UniqueName: \"kubernetes.io/projected/ad723ec0-e63c-4a5e-acda-719533071ba2-kube-api-access-f65f5\") pod \"limitador-operator-controller-manager-859d7b78bd-xj2dd\" (UID: \"ad723ec0-e63c-4a5e-acda-719533071ba2\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xj2dd"
Jul 21 12:48:46.113958 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:48:46.113872    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/560dca78-8e2e-4eb5-a8fe-1915d354e0c5-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-wcvml\" (UID: \"560dca78-8e2e-4eb5-a8fe-1915d354e0c5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wcvml"
Jul 21 12:48:46.113996 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:48:46.113965    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zcrcn\" (UniqueName: \"kubernetes.io/projected/560dca78-8e2e-4eb5-a8fe-1915d354e0c5-kube-api-access-zcrcn\") pod \"kuadrant-operator-controller-manager-8684c9ff49-wcvml\" (UID: \"560dca78-8e2e-4eb5-a8fe-1915d354e0c5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wcvml"
Jul 21 12:48:46.114196 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:48:46.114178    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/560dca78-8e2e-4eb5-a8fe-1915d354e0c5-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-wcvml\" (UID: \"560dca78-8e2e-4eb5-a8fe-1915d354e0c5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wcvml"
Jul 21 12:48:46.123234 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:48:46.123213    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zcrcn\" (UniqueName: \"kubernetes.io/projected/560dca78-8e2e-4eb5-a8fe-1915d354e0c5-kube-api-access-zcrcn\") pod \"kuadrant-operator-controller-manager-8684c9ff49-wcvml\" (UID: \"560dca78-8e2e-4eb5-a8fe-1915d354e0c5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wcvml"
Jul 21 12:48:46.123321 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:48:46.123213    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f65f5\" (UniqueName: \"kubernetes.io/projected/ad723ec0-e63c-4a5e-acda-719533071ba2-kube-api-access-f65f5\") pod \"limitador-operator-controller-manager-859d7b78bd-xj2dd\" (UID: \"ad723ec0-e63c-4a5e-acda-719533071ba2\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xj2dd"
Jul 21 12:48:46.214475 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:48:46.214441    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gxxlb\" (UniqueName: \"kubernetes.io/projected/9be9eed0-62ad-4462-b48a-3629d9c15bd0-kube-api-access-gxxlb\") pod \"9be9eed0-62ad-4462-b48a-3629d9c15bd0\" (UID: \"9be9eed0-62ad-4462-b48a-3629d9c15bd0\") "
Jul 21 12:48:46.214646 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:48:46.214498    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r94ts\" (UniqueName: \"kubernetes.io/projected/afd3056a-3199-4492-abf6-8aa8afb611cb-kube-api-access-r94ts\") pod \"afd3056a-3199-4492-abf6-8aa8afb611cb\" (UID: \"afd3056a-3199-4492-abf6-8aa8afb611cb\") "
Jul 21 12:48:46.214702 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:48:46.214681    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/afd3056a-3199-4492-abf6-8aa8afb611cb-extensions-socket-volume\") pod \"afd3056a-3199-4492-abf6-8aa8afb611cb\" (UID: \"afd3056a-3199-4492-abf6-8aa8afb611cb\") "
Jul 21 12:48:46.215246 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:48:46.215173    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/afd3056a-3199-4492-abf6-8aa8afb611cb-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "afd3056a-3199-4492-abf6-8aa8afb611cb" (UID: "afd3056a-3199-4492-abf6-8aa8afb611cb"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 12:48:46.216694 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:48:46.216672    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/afd3056a-3199-4492-abf6-8aa8afb611cb-kube-api-access-r94ts" (OuterVolumeSpecName: "kube-api-access-r94ts") pod "afd3056a-3199-4492-abf6-8aa8afb611cb" (UID: "afd3056a-3199-4492-abf6-8aa8afb611cb"). InnerVolumeSpecName "kube-api-access-r94ts". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 12:48:46.216799 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:48:46.216724    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9be9eed0-62ad-4462-b48a-3629d9c15bd0-kube-api-access-gxxlb" (OuterVolumeSpecName: "kube-api-access-gxxlb") pod "9be9eed0-62ad-4462-b48a-3629d9c15bd0" (UID: "9be9eed0-62ad-4462-b48a-3629d9c15bd0"). InnerVolumeSpecName "kube-api-access-gxxlb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 12:48:46.233031 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:48:46.232993    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wcvml"
Jul 21 12:48:46.238082 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:48:46.238065    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xj2dd"
Jul 21 12:48:46.318528 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:48:46.317875    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r94ts\" (UniqueName: \"kubernetes.io/projected/afd3056a-3199-4492-abf6-8aa8afb611cb-kube-api-access-r94ts\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Jul 21 12:48:46.318528 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:48:46.317910    2556 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/afd3056a-3199-4492-abf6-8aa8afb611cb-extensions-socket-volume\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Jul 21 12:48:46.318528 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:48:46.317925    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gxxlb\" (UniqueName: \"kubernetes.io/projected/9be9eed0-62ad-4462-b48a-3629d9c15bd0-kube-api-access-gxxlb\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Jul 21 12:48:46.432085 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:48:46.432060    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xj2dd"]
Jul 21 12:48:46.434183 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:48:46.434138    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad723ec0_e63c_4a5e_acda_719533071ba2.slice/crio-f757789cdfc6225d1857aca272d81ef4d4fd2795223e5991daa542b0ea91a7f7 WatchSource:0}: Error finding container f757789cdfc6225d1857aca272d81ef4d4fd2795223e5991daa542b0ea91a7f7: Status 404 returned error can't find the container with id f757789cdfc6225d1857aca272d81ef4d4fd2795223e5991daa542b0ea91a7f7
Jul 21 12:48:46.434291 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:48:46.434246    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wcvml"]
Jul 21 12:48:46.436001 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:48:46.435975    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod560dca78_8e2e_4eb5_a8fe_1915d354e0c5.slice/crio-5771a5cf7fdbdacba17c8f7a3398b71d8d62291e42ba87249c6a5bbc933e621f WatchSource:0}: Error finding container 5771a5cf7fdbdacba17c8f7a3398b71d8d62291e42ba87249c6a5bbc933e621f: Status 404 returned error can't find the container with id 5771a5cf7fdbdacba17c8f7a3398b71d8d62291e42ba87249c6a5bbc933e621f
Jul 21 12:48:46.593285 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:48:46.593249    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-pmr2h"
Jul 21 12:48:46.631612 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:48:46.631510    2556 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-mqms8\" is forbidden: User \"system:node:ip-10-0-130-2.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-2.ec2.internal' and this object" podUID="afd3056a-3199-4492-abf6-8aa8afb611cb" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mqms8"
Jul 21 12:48:46.633981 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:48:46.633957    2556 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-9wtcz\" is forbidden: User \"system:node:ip-10-0-130-2.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-2.ec2.internal' and this object" podUID="9be9eed0-62ad-4462-b48a-3629d9c15bd0" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9wtcz"
Jul 21 12:48:46.653056 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:48:46.652994    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xj2dd" event={"ID":"ad723ec0-e63c-4a5e-acda-719533071ba2","Type":"ContainerStarted","Data":"bdb8d50fbe7d5b0836b29178ba988824e35ce1affce27229c8e203e1eba2759d"}
Jul 21 12:48:46.653203 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:48:46.653124    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xj2dd" event={"ID":"ad723ec0-e63c-4a5e-acda-719533071ba2","Type":"ContainerStarted","Data":"f757789cdfc6225d1857aca272d81ef4d4fd2795223e5991daa542b0ea91a7f7"}
Jul 21 12:48:46.653361 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:48:46.653335    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xj2dd"
Jul 21 12:48:46.654306 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:48:46.654284    2556 generic.go:356] "Generic (PLEG): container finished" podID="9be9eed0-62ad-4462-b48a-3629d9c15bd0" containerID="b391f3a1256bab4462b0042efe6d9a42ac463a194cdcdf0bb30622d753b2df40" exitCode=0
Jul 21 12:48:46.654430 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:48:46.654348    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9wtcz"
Jul 21 12:48:46.654430 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:48:46.654383    2556 scope.go:117] "RemoveContainer" containerID="b391f3a1256bab4462b0042efe6d9a42ac463a194cdcdf0bb30622d753b2df40"
Jul 21 12:48:46.655465 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:48:46.655375    2556 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-mqms8\" is forbidden: User \"system:node:ip-10-0-130-2.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-2.ec2.internal' and this object" podUID="afd3056a-3199-4492-abf6-8aa8afb611cb" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mqms8"
Jul 21 12:48:46.655758 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:48:46.655737    2556 generic.go:356] "Generic (PLEG): container finished" podID="afd3056a-3199-4492-abf6-8aa8afb611cb" containerID="c39edb1df27fcead1233f510d9365ccd95f602eec481f9926705cdbda922540e" exitCode=0
Jul 21 12:48:46.655875 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:48:46.655862    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mqms8"
Jul 21 12:48:46.657741 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:48:46.657709    2556 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-9wtcz\" is forbidden: User \"system:node:ip-10-0-130-2.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-2.ec2.internal' and this object" podUID="9be9eed0-62ad-4462-b48a-3629d9c15bd0" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9wtcz"
Jul 21 12:48:46.658883 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:48:46.658861    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wcvml" event={"ID":"560dca78-8e2e-4eb5-a8fe-1915d354e0c5","Type":"ContainerStarted","Data":"b39580a14f67f7b6f27fbe0ca740125378432be38710d7b6790a75b67c1d3f37"}
Jul 21 12:48:46.659084 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:48:46.659060    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wcvml" event={"ID":"560dca78-8e2e-4eb5-a8fe-1915d354e0c5","Type":"ContainerStarted","Data":"5771a5cf7fdbdacba17c8f7a3398b71d8d62291e42ba87249c6a5bbc933e621f"}
Jul 21 12:48:46.659186 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:48:46.659103    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wcvml"
Jul 21 12:48:46.665139 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:48:46.665121    2556 scope.go:117] "RemoveContainer" containerID="b391f3a1256bab4462b0042efe6d9a42ac463a194cdcdf0bb30622d753b2df40"
Jul 21 12:48:46.665576 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:48:46.665549    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b391f3a1256bab4462b0042efe6d9a42ac463a194cdcdf0bb30622d753b2df40\": container with ID starting with b391f3a1256bab4462b0042efe6d9a42ac463a194cdcdf0bb30622d753b2df40 not found: ID does not exist" containerID="b391f3a1256bab4462b0042efe6d9a42ac463a194cdcdf0bb30622d753b2df40"
Jul 21 12:48:46.665659 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:48:46.665589    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b391f3a1256bab4462b0042efe6d9a42ac463a194cdcdf0bb30622d753b2df40"} err="failed to get container status \"b391f3a1256bab4462b0042efe6d9a42ac463a194cdcdf0bb30622d753b2df40\": rpc error: code = NotFound desc = could not find container \"b391f3a1256bab4462b0042efe6d9a42ac463a194cdcdf0bb30622d753b2df40\": container with ID starting with b391f3a1256bab4462b0042efe6d9a42ac463a194cdcdf0bb30622d753b2df40 not found: ID does not exist"
Jul 21 12:48:46.665659 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:48:46.665610    2556 scope.go:117] "RemoveContainer" containerID="c39edb1df27fcead1233f510d9365ccd95f602eec481f9926705cdbda922540e"
Jul 21 12:48:46.675154 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:48:46.675135    2556 scope.go:117] "RemoveContainer" containerID="c39edb1df27fcead1233f510d9365ccd95f602eec481f9926705cdbda922540e"
Jul 21 12:48:46.675429 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:48:46.675408    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c39edb1df27fcead1233f510d9365ccd95f602eec481f9926705cdbda922540e\": container with ID starting with c39edb1df27fcead1233f510d9365ccd95f602eec481f9926705cdbda922540e not found: ID does not exist" containerID="c39edb1df27fcead1233f510d9365ccd95f602eec481f9926705cdbda922540e"
Jul 21 12:48:46.675497 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:48:46.675439    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c39edb1df27fcead1233f510d9365ccd95f602eec481f9926705cdbda922540e"} err="failed to get container status \"c39edb1df27fcead1233f510d9365ccd95f602eec481f9926705cdbda922540e\": rpc error: code = NotFound desc = could not find container \"c39edb1df27fcead1233f510d9365ccd95f602eec481f9926705cdbda922540e\": container with ID starting with c39edb1df27fcead1233f510d9365ccd95f602eec481f9926705cdbda922540e not found: ID does not exist"
Jul 21 12:48:46.682801 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:48:46.682759    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xj2dd" podStartSLOduration=1.682746657 podStartE2EDuration="1.682746657s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:48:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:48:46.680816789 +0000 UTC m=+539.984821879" watchObservedRunningTime="2026-07-21 12:48:46.682746657 +0000 UTC m=+539.986751748"
Jul 21 12:48:46.683076 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:48:46.683050    2556 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-mqms8\" is forbidden: User \"system:node:ip-10-0-130-2.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-2.ec2.internal' and this object" podUID="afd3056a-3199-4492-abf6-8aa8afb611cb" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mqms8"
Jul 21 12:48:46.684669 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:48:46.684641    2556 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-9wtcz\" is forbidden: User \"system:node:ip-10-0-130-2.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-2.ec2.internal' and this object" podUID="9be9eed0-62ad-4462-b48a-3629d9c15bd0" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9wtcz"
Jul 21 12:48:46.711610 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:48:46.711553    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wcvml" podStartSLOduration=1.711532627 podStartE2EDuration="1.711532627s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:48:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:48:46.710853835 +0000 UTC m=+540.014858924" watchObservedRunningTime="2026-07-21 12:48:46.711532627 +0000 UTC m=+540.015537718"
Jul 21 12:48:47.192652 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:48:47.192597    2556 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-mqms8\" is forbidden: User \"system:node:ip-10-0-130-2.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-2.ec2.internal' and this object" podUID="afd3056a-3199-4492-abf6-8aa8afb611cb" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mqms8"
Jul 21 12:48:47.195730 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:48:47.195703    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9be9eed0-62ad-4462-b48a-3629d9c15bd0" path="/var/lib/kubelet/pods/9be9eed0-62ad-4462-b48a-3629d9c15bd0/volumes"
Jul 21 12:48:47.196108 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:48:47.196095    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="afd3056a-3199-4492-abf6-8aa8afb611cb" path="/var/lib/kubelet/pods/afd3056a-3199-4492-abf6-8aa8afb611cb/volumes"
Jul 21 12:48:47.196664 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:48:47.196642    2556 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-9wtcz\" is forbidden: User \"system:node:ip-10-0-130-2.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-2.ec2.internal' and this object" podUID="9be9eed0-62ad-4462-b48a-3629d9c15bd0" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9wtcz"
Jul 21 12:48:57.666461 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:48:57.666422    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wcvml"
Jul 21 12:48:57.667030 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:48:57.666621    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xj2dd"
Jul 21 12:49:04.603045 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:04.602999    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wcvml"]
Jul 21 12:49:04.603413 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:04.603201    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wcvml" podUID="560dca78-8e2e-4eb5-a8fe-1915d354e0c5" containerName="manager" containerID="cri-o://b39580a14f67f7b6f27fbe0ca740125378432be38710d7b6790a75b67c1d3f37" gracePeriod=10
Jul 21 12:49:04.752419 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:04.752385    2556 generic.go:356] "Generic (PLEG): container finished" podID="560dca78-8e2e-4eb5-a8fe-1915d354e0c5" containerID="b39580a14f67f7b6f27fbe0ca740125378432be38710d7b6790a75b67c1d3f37" exitCode=0
Jul 21 12:49:04.752564 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:04.752445    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wcvml" event={"ID":"560dca78-8e2e-4eb5-a8fe-1915d354e0c5","Type":"ContainerDied","Data":"b39580a14f67f7b6f27fbe0ca740125378432be38710d7b6790a75b67c1d3f37"}
Jul 21 12:49:04.848459 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:04.848437    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wcvml"
Jul 21 12:49:04.876102 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:04.875995    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zcrcn\" (UniqueName: \"kubernetes.io/projected/560dca78-8e2e-4eb5-a8fe-1915d354e0c5-kube-api-access-zcrcn\") pod \"560dca78-8e2e-4eb5-a8fe-1915d354e0c5\" (UID: \"560dca78-8e2e-4eb5-a8fe-1915d354e0c5\") "
Jul 21 12:49:04.876220 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:04.876121    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/560dca78-8e2e-4eb5-a8fe-1915d354e0c5-extensions-socket-volume\") pod \"560dca78-8e2e-4eb5-a8fe-1915d354e0c5\" (UID: \"560dca78-8e2e-4eb5-a8fe-1915d354e0c5\") "
Jul 21 12:49:04.876740 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:04.876706    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/560dca78-8e2e-4eb5-a8fe-1915d354e0c5-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "560dca78-8e2e-4eb5-a8fe-1915d354e0c5" (UID: "560dca78-8e2e-4eb5-a8fe-1915d354e0c5"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 12:49:04.878593 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:04.878562    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/560dca78-8e2e-4eb5-a8fe-1915d354e0c5-kube-api-access-zcrcn" (OuterVolumeSpecName: "kube-api-access-zcrcn") pod "560dca78-8e2e-4eb5-a8fe-1915d354e0c5" (UID: "560dca78-8e2e-4eb5-a8fe-1915d354e0c5"). InnerVolumeSpecName "kube-api-access-zcrcn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 12:49:04.977619 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:04.977597    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zcrcn\" (UniqueName: \"kubernetes.io/projected/560dca78-8e2e-4eb5-a8fe-1915d354e0c5-kube-api-access-zcrcn\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Jul 21 12:49:04.977619 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:04.977618    2556 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/560dca78-8e2e-4eb5-a8fe-1915d354e0c5-extensions-socket-volume\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Jul 21 12:49:05.758258 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:05.758233    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wcvml"
Jul 21 12:49:05.758659 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:05.758278    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wcvml" event={"ID":"560dca78-8e2e-4eb5-a8fe-1915d354e0c5","Type":"ContainerDied","Data":"5771a5cf7fdbdacba17c8f7a3398b71d8d62291e42ba87249c6a5bbc933e621f"}
Jul 21 12:49:05.758659 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:05.758333    2556 scope.go:117] "RemoveContainer" containerID="b39580a14f67f7b6f27fbe0ca740125378432be38710d7b6790a75b67c1d3f37"
Jul 21 12:49:05.791657 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:05.791624    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wcvml"]
Jul 21 12:49:05.797212 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:05.797186    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wcvml"]
Jul 21 12:49:07.194884 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:07.194849    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="560dca78-8e2e-4eb5-a8fe-1915d354e0c5" path="/var/lib/kubelet/pods/560dca78-8e2e-4eb5-a8fe-1915d354e0c5/volumes"
Jul 21 12:49:21.205583 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:21.205539    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-b55kr"]
Jul 21 12:49:21.206181 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:21.206159    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="560dca78-8e2e-4eb5-a8fe-1915d354e0c5" containerName="manager"
Jul 21 12:49:21.206181 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:21.206179    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="560dca78-8e2e-4eb5-a8fe-1915d354e0c5" containerName="manager"
Jul 21 12:49:21.206311 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:21.206280    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="560dca78-8e2e-4eb5-a8fe-1915d354e0c5" containerName="manager"
Jul 21 12:49:21.213559 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:21.213530    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-b55kr"
Jul 21 12:49:21.218193 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:21.218170    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-dlvv8\""
Jul 21 12:49:21.225789 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:21.225766    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-b55kr"]
Jul 21 12:49:21.317263 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:21.317228    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fe4ed9c9-5046-4bb2-acbd-8d83d8b8eb2c-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-b55kr\" (UID: \"fe4ed9c9-5046-4bb2-acbd-8d83d8b8eb2c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-b55kr"
Jul 21 12:49:21.317448 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:21.317268    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fe4ed9c9-5046-4bb2-acbd-8d83d8b8eb2c-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-b55kr\" (UID: \"fe4ed9c9-5046-4bb2-acbd-8d83d8b8eb2c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-b55kr"
Jul 21 12:49:21.317448 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:21.317298    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9xbh9\" (UniqueName: \"kubernetes.io/projected/fe4ed9c9-5046-4bb2-acbd-8d83d8b8eb2c-kube-api-access-9xbh9\") pod \"maas-default-gateway-openshift-default-56654dcf7-b55kr\" (UID: \"fe4ed9c9-5046-4bb2-acbd-8d83d8b8eb2c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-b55kr"
Jul 21 12:49:21.317539 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:21.317454    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fe4ed9c9-5046-4bb2-acbd-8d83d8b8eb2c-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-b55kr\" (UID: \"fe4ed9c9-5046-4bb2-acbd-8d83d8b8eb2c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-b55kr"
Jul 21 12:49:21.317539 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:21.317489    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fe4ed9c9-5046-4bb2-acbd-8d83d8b8eb2c-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-b55kr\" (UID: \"fe4ed9c9-5046-4bb2-acbd-8d83d8b8eb2c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-b55kr"
Jul 21 12:49:21.317539 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:21.317509    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe4ed9c9-5046-4bb2-acbd-8d83d8b8eb2c-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-b55kr\" (UID: \"fe4ed9c9-5046-4bb2-acbd-8d83d8b8eb2c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-b55kr"
Jul 21 12:49:21.317648 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:21.317594    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe4ed9c9-5046-4bb2-acbd-8d83d8b8eb2c-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-b55kr\" (UID: \"fe4ed9c9-5046-4bb2-acbd-8d83d8b8eb2c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-b55kr"
Jul 21 12:49:21.317648 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:21.317620    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fe4ed9c9-5046-4bb2-acbd-8d83d8b8eb2c-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-b55kr\" (UID: \"fe4ed9c9-5046-4bb2-acbd-8d83d8b8eb2c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-b55kr"
Jul 21 12:49:21.317648 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:21.317640    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fe4ed9c9-5046-4bb2-acbd-8d83d8b8eb2c-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-b55kr\" (UID: \"fe4ed9c9-5046-4bb2-acbd-8d83d8b8eb2c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-b55kr"
Jul 21 12:49:21.419108 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:21.419073    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fe4ed9c9-5046-4bb2-acbd-8d83d8b8eb2c-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-b55kr\" (UID: \"fe4ed9c9-5046-4bb2-acbd-8d83d8b8eb2c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-b55kr"
Jul 21 12:49:21.419108 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:21.419114    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fe4ed9c9-5046-4bb2-acbd-8d83d8b8eb2c-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-b55kr\" (UID: \"fe4ed9c9-5046-4bb2-acbd-8d83d8b8eb2c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-b55kr"
Jul 21 12:49:21.419345 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:21.419139    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9xbh9\" (UniqueName: \"kubernetes.io/projected/fe4ed9c9-5046-4bb2-acbd-8d83d8b8eb2c-kube-api-access-9xbh9\") pod \"maas-default-gateway-openshift-default-56654dcf7-b55kr\" (UID: \"fe4ed9c9-5046-4bb2-acbd-8d83d8b8eb2c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-b55kr"
Jul 21 12:49:21.419345 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:21.419199    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fe4ed9c9-5046-4bb2-acbd-8d83d8b8eb2c-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-b55kr\" (UID: \"fe4ed9c9-5046-4bb2-acbd-8d83d8b8eb2c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-b55kr"
Jul 21 12:49:21.419345 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:21.419219    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fe4ed9c9-5046-4bb2-acbd-8d83d8b8eb2c-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-b55kr\" (UID: \"fe4ed9c9-5046-4bb2-acbd-8d83d8b8eb2c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-b55kr"
Jul 21 12:49:21.419345 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:21.419240    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe4ed9c9-5046-4bb2-acbd-8d83d8b8eb2c-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-b55kr\" (UID: \"fe4ed9c9-5046-4bb2-acbd-8d83d8b8eb2c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-b55kr"
Jul 21 12:49:21.419345 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:21.419274    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe4ed9c9-5046-4bb2-acbd-8d83d8b8eb2c-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-b55kr\" (UID: \"fe4ed9c9-5046-4bb2-acbd-8d83d8b8eb2c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-b55kr"
Jul 21 12:49:21.419345 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:21.419298    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fe4ed9c9-5046-4bb2-acbd-8d83d8b8eb2c-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-b55kr\" (UID: \"fe4ed9c9-5046-4bb2-acbd-8d83d8b8eb2c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-b55kr"
Jul 21 12:49:21.419345 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:21.419325    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fe4ed9c9-5046-4bb2-acbd-8d83d8b8eb2c-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-b55kr\" (UID: \"fe4ed9c9-5046-4bb2-acbd-8d83d8b8eb2c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-b55kr"
Jul 21 12:49:21.419752 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:21.419713    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe4ed9c9-5046-4bb2-acbd-8d83d8b8eb2c-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-b55kr\" (UID: \"fe4ed9c9-5046-4bb2-acbd-8d83d8b8eb2c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-b55kr"
Jul 21 12:49:21.419752 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:21.419726    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fe4ed9c9-5046-4bb2-acbd-8d83d8b8eb2c-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-b55kr\" (UID: \"fe4ed9c9-5046-4bb2-acbd-8d83d8b8eb2c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-b55kr"
Jul 21 12:49:21.419936 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:21.419916    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fe4ed9c9-5046-4bb2-acbd-8d83d8b8eb2c-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-b55kr\" (UID: \"fe4ed9c9-5046-4bb2-acbd-8d83d8b8eb2c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-b55kr"
Jul 21 12:49:21.420514 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:21.420486    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe4ed9c9-5046-4bb2-acbd-8d83d8b8eb2c-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-b55kr\" (UID: \"fe4ed9c9-5046-4bb2-acbd-8d83d8b8eb2c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-b55kr"
Jul 21 12:49:21.420973 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:21.420839    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fe4ed9c9-5046-4bb2-acbd-8d83d8b8eb2c-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-b55kr\" (UID: \"fe4ed9c9-5046-4bb2-acbd-8d83d8b8eb2c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-b55kr"
Jul 21 12:49:21.421958 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:21.421931    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fe4ed9c9-5046-4bb2-acbd-8d83d8b8eb2c-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-b55kr\" (UID: \"fe4ed9c9-5046-4bb2-acbd-8d83d8b8eb2c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-b55kr"
Jul 21 12:49:21.422124 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:21.422104    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fe4ed9c9-5046-4bb2-acbd-8d83d8b8eb2c-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-b55kr\" (UID: \"fe4ed9c9-5046-4bb2-acbd-8d83d8b8eb2c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-b55kr"
Jul 21 12:49:21.437318 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:21.437287    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fe4ed9c9-5046-4bb2-acbd-8d83d8b8eb2c-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-b55kr\" (UID: \"fe4ed9c9-5046-4bb2-acbd-8d83d8b8eb2c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-b55kr"
Jul 21 12:49:21.437603 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:21.437585    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9xbh9\" (UniqueName: \"kubernetes.io/projected/fe4ed9c9-5046-4bb2-acbd-8d83d8b8eb2c-kube-api-access-9xbh9\") pod \"maas-default-gateway-openshift-default-56654dcf7-b55kr\" (UID: \"fe4ed9c9-5046-4bb2-acbd-8d83d8b8eb2c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-b55kr"
Jul 21 12:49:21.525195 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:21.525168    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-b55kr"
Jul 21 12:49:21.689445 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:21.689420    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-b55kr"]
Jul 21 12:49:21.691912 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:49:21.691885    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe4ed9c9_5046_4bb2_acbd_8d83d8b8eb2c.slice/crio-3e478ffac3e1d0fa5bf1c915162e04fabb116991bff1a753242e0f6d691061ef WatchSource:0}: Error finding container 3e478ffac3e1d0fa5bf1c915162e04fabb116991bff1a753242e0f6d691061ef: Status 404 returned error can't find the container with id 3e478ffac3e1d0fa5bf1c915162e04fabb116991bff1a753242e0f6d691061ef
Jul 21 12:49:21.693969 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:21.693941    2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 21 12:49:21.694050 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:21.694008    2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 21 12:49:21.694092 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:21.694076    2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 21 12:49:21.845190 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:21.845109    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-b55kr" event={"ID":"fe4ed9c9-5046-4bb2-acbd-8d83d8b8eb2c","Type":"ContainerStarted","Data":"3244c07f82a77cafd767cfa4e6a844ac4492dae67ff128ebe549bbf7fcf6ef67"}
Jul 21 12:49:21.845190 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:21.845168    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-b55kr" event={"ID":"fe4ed9c9-5046-4bb2-acbd-8d83d8b8eb2c","Type":"ContainerStarted","Data":"3e478ffac3e1d0fa5bf1c915162e04fabb116991bff1a753242e0f6d691061ef"}
Jul 21 12:49:21.878428 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:21.878367    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-b55kr" podStartSLOduration=0.878346759 podStartE2EDuration="878.346759ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:49:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:49:21.876810386 +0000 UTC m=+575.180815475" watchObservedRunningTime="2026-07-21 12:49:21.878346759 +0000 UTC m=+575.182351850"
Jul 21 12:49:22.525656 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:22.525625    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-b55kr"
Jul 21 12:49:22.530479 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:22.530451    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-b55kr"
Jul 21 12:49:22.850873 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:22.850801    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-b55kr"
Jul 21 12:49:22.851822 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:22.851798    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-b55kr"
Jul 21 12:49:23.283081 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:23.283048    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-79f7t"]
Jul 21 12:49:23.286550 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:23.286534    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-79f7t"
Jul 21 12:49:23.289334 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:23.289313    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-hsjs2\""
Jul 21 12:49:23.289923 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:23.289894    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 21 12:49:23.298203 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:23.298182    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-79f7t"]
Jul 21 12:49:23.336691 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:23.336660    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-79f7t"]
Jul 21 12:49:23.439288 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:23.439253    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-98g8v\" (UniqueName: \"kubernetes.io/projected/5f88a56b-cd09-4b29-8a2e-eb2c8a4ecf9c-kube-api-access-98g8v\") pod \"limitador-limitador-64c94fd64f-79f7t\" (UID: \"5f88a56b-cd09-4b29-8a2e-eb2c8a4ecf9c\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-79f7t"
Jul 21 12:49:23.439458 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:23.439339    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/5f88a56b-cd09-4b29-8a2e-eb2c8a4ecf9c-config-file\") pod \"limitador-limitador-64c94fd64f-79f7t\" (UID: \"5f88a56b-cd09-4b29-8a2e-eb2c8a4ecf9c\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-79f7t"
Jul 21 12:49:23.540527 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:23.540440    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-98g8v\" (UniqueName: \"kubernetes.io/projected/5f88a56b-cd09-4b29-8a2e-eb2c8a4ecf9c-kube-api-access-98g8v\") pod \"limitador-limitador-64c94fd64f-79f7t\" (UID: \"5f88a56b-cd09-4b29-8a2e-eb2c8a4ecf9c\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-79f7t"
Jul 21 12:49:23.540527 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:23.540515    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/5f88a56b-cd09-4b29-8a2e-eb2c8a4ecf9c-config-file\") pod \"limitador-limitador-64c94fd64f-79f7t\" (UID: \"5f88a56b-cd09-4b29-8a2e-eb2c8a4ecf9c\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-79f7t"
Jul 21 12:49:23.541207 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:23.541189    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/5f88a56b-cd09-4b29-8a2e-eb2c8a4ecf9c-config-file\") pod \"limitador-limitador-64c94fd64f-79f7t\" (UID: \"5f88a56b-cd09-4b29-8a2e-eb2c8a4ecf9c\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-79f7t"
Jul 21 12:49:23.558714 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:23.558685    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-98g8v\" (UniqueName: \"kubernetes.io/projected/5f88a56b-cd09-4b29-8a2e-eb2c8a4ecf9c-kube-api-access-98g8v\") pod \"limitador-limitador-64c94fd64f-79f7t\" (UID: \"5f88a56b-cd09-4b29-8a2e-eb2c8a4ecf9c\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-79f7t"
Jul 21 12:49:23.597430 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:23.597392    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-79f7t"
Jul 21 12:49:23.785953 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:23.785920    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-79f7t"]
Jul 21 12:49:23.789912 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:49:23.789885    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f88a56b_cd09_4b29_8a2e_eb2c8a4ecf9c.slice/crio-5e93dd9b3496ddf66e900e7d45a139147834ee883e063ae8a04f07d1a522b1a8 WatchSource:0}: Error finding container 5e93dd9b3496ddf66e900e7d45a139147834ee883e063ae8a04f07d1a522b1a8: Status 404 returned error can't find the container with id 5e93dd9b3496ddf66e900e7d45a139147834ee883e063ae8a04f07d1a522b1a8
Jul 21 12:49:23.857024 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:23.856981    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-79f7t" event={"ID":"5f88a56b-cd09-4b29-8a2e-eb2c8a4ecf9c","Type":"ContainerStarted","Data":"5e93dd9b3496ddf66e900e7d45a139147834ee883e063ae8a04f07d1a522b1a8"}
Jul 21 12:49:24.356901 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:24.356876    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-rqktz"]
Jul 21 12:49:24.361496 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:24.361477    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-rqktz"
Jul 21 12:49:24.365362 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:24.365341    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-m9vxc\""
Jul 21 12:49:24.371842 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:24.371822    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-rqktz"]
Jul 21 12:49:24.448873 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:24.448832    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fjt8g\" (UniqueName: \"kubernetes.io/projected/b0b32ced-e1af-4c19-8f93-e27942b67c37-kube-api-access-fjt8g\") pod \"authorino-58cd955675-rqktz\" (UID: \"b0b32ced-e1af-4c19-8f93-e27942b67c37\") " pod="kuadrant-system/authorino-58cd955675-rqktz"
Jul 21 12:49:24.550171 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:24.550147    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fjt8g\" (UniqueName: \"kubernetes.io/projected/b0b32ced-e1af-4c19-8f93-e27942b67c37-kube-api-access-fjt8g\") pod \"authorino-58cd955675-rqktz\" (UID: \"b0b32ced-e1af-4c19-8f93-e27942b67c37\") " pod="kuadrant-system/authorino-58cd955675-rqktz"
Jul 21 12:49:24.565433 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:24.565409    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fjt8g\" (UniqueName: \"kubernetes.io/projected/b0b32ced-e1af-4c19-8f93-e27942b67c37-kube-api-access-fjt8g\") pod \"authorino-58cd955675-rqktz\" (UID: \"b0b32ced-e1af-4c19-8f93-e27942b67c37\") " pod="kuadrant-system/authorino-58cd955675-rqktz"
Jul 21 12:49:24.671441 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:24.671355    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-rqktz"
Jul 21 12:49:24.869619 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:24.869553    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-rqktz"]
Jul 21 12:49:25.873442 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:25.873408    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-rqktz" event={"ID":"b0b32ced-e1af-4c19-8f93-e27942b67c37","Type":"ContainerStarted","Data":"8b45b7277799e0dfc5d1b14a8d5f4c91a5475639e4569064dfbb2104ec89572f"}
Jul 21 12:49:26.883353 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:26.883285    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-79f7t" event={"ID":"5f88a56b-cd09-4b29-8a2e-eb2c8a4ecf9c","Type":"ContainerStarted","Data":"99d8232341e63a47e94bbf1f18d60c290a779f9cd593012c0423805df6da513f"}
Jul 21 12:49:26.884002 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:26.883953    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-79f7t"
Jul 21 12:49:26.910023 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:26.909965    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-79f7t" podStartSLOduration=0.98762819 podStartE2EDuration="3.909945838s" totalImagesPullingTime="2.922317648s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:49:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:49:26.907084335 +0000 UTC m=+580.211089632" watchObservedRunningTime="2026-07-21 12:49:26.909945838 +0000 UTC m=+580.213950929"
Jul 21 12:49:28.895329 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:28.895284    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-rqktz" event={"ID":"b0b32ced-e1af-4c19-8f93-e27942b67c37","Type":"ContainerStarted","Data":"4087a72a6a3046d4ef5229ea9df1574b8df9c391abeb1301b7ba8bcd03ba2881"}
Jul 21 12:49:28.914490 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:28.914416    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-rqktz" podStartSLOduration=1.004683065 podStartE2EDuration="4.914394307s" totalImagesPullingTime="3.909711242s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:49:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:49:28.913455299 +0000 UTC m=+582.217460390" watchObservedRunningTime="2026-07-21 12:49:28.914394307 +0000 UTC m=+582.218399398"
Jul 21 12:49:29.499290 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:29.499253    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-rqktz"]
Jul 21 12:49:30.904992 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:30.904948    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-rqktz" podUID="b0b32ced-e1af-4c19-8f93-e27942b67c37" containerName="authorino" containerID="cri-o://4087a72a6a3046d4ef5229ea9df1574b8df9c391abeb1301b7ba8bcd03ba2881" gracePeriod=30
Jul 21 12:49:31.160968 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:31.160913    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-rqktz"
Jul 21 12:49:31.311611 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:31.311586    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fjt8g\" (UniqueName: \"kubernetes.io/projected/b0b32ced-e1af-4c19-8f93-e27942b67c37-kube-api-access-fjt8g\") pod \"b0b32ced-e1af-4c19-8f93-e27942b67c37\" (UID: \"b0b32ced-e1af-4c19-8f93-e27942b67c37\") "
Jul 21 12:49:31.313639 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:31.313617    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b0b32ced-e1af-4c19-8f93-e27942b67c37-kube-api-access-fjt8g" (OuterVolumeSpecName: "kube-api-access-fjt8g") pod "b0b32ced-e1af-4c19-8f93-e27942b67c37" (UID: "b0b32ced-e1af-4c19-8f93-e27942b67c37"). InnerVolumeSpecName "kube-api-access-fjt8g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 12:49:31.413116 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:31.413061    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fjt8g\" (UniqueName: \"kubernetes.io/projected/b0b32ced-e1af-4c19-8f93-e27942b67c37-kube-api-access-fjt8g\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Jul 21 12:49:31.911056 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:31.911001    2556 generic.go:356] "Generic (PLEG): container finished" podID="b0b32ced-e1af-4c19-8f93-e27942b67c37" containerID="4087a72a6a3046d4ef5229ea9df1574b8df9c391abeb1301b7ba8bcd03ba2881" exitCode=0
Jul 21 12:49:31.911056 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:31.911042    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-rqktz" event={"ID":"b0b32ced-e1af-4c19-8f93-e27942b67c37","Type":"ContainerDied","Data":"4087a72a6a3046d4ef5229ea9df1574b8df9c391abeb1301b7ba8bcd03ba2881"}
Jul 21 12:49:31.911499 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:31.911062    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-rqktz"
Jul 21 12:49:31.911499 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:31.911088    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-rqktz" event={"ID":"b0b32ced-e1af-4c19-8f93-e27942b67c37","Type":"ContainerDied","Data":"8b45b7277799e0dfc5d1b14a8d5f4c91a5475639e4569064dfbb2104ec89572f"}
Jul 21 12:49:31.911499 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:31.911107    2556 scope.go:117] "RemoveContainer" containerID="4087a72a6a3046d4ef5229ea9df1574b8df9c391abeb1301b7ba8bcd03ba2881"
Jul 21 12:49:31.919709 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:31.919692    2556 scope.go:117] "RemoveContainer" containerID="4087a72a6a3046d4ef5229ea9df1574b8df9c391abeb1301b7ba8bcd03ba2881"
Jul 21 12:49:31.919939 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:49:31.919923    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4087a72a6a3046d4ef5229ea9df1574b8df9c391abeb1301b7ba8bcd03ba2881\": container with ID starting with 4087a72a6a3046d4ef5229ea9df1574b8df9c391abeb1301b7ba8bcd03ba2881 not found: ID does not exist" containerID="4087a72a6a3046d4ef5229ea9df1574b8df9c391abeb1301b7ba8bcd03ba2881"
Jul 21 12:49:31.920002 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:31.919946    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4087a72a6a3046d4ef5229ea9df1574b8df9c391abeb1301b7ba8bcd03ba2881"} err="failed to get container status \"4087a72a6a3046d4ef5229ea9df1574b8df9c391abeb1301b7ba8bcd03ba2881\": rpc error: code = NotFound desc = could not find container \"4087a72a6a3046d4ef5229ea9df1574b8df9c391abeb1301b7ba8bcd03ba2881\": container with ID starting with 4087a72a6a3046d4ef5229ea9df1574b8df9c391abeb1301b7ba8bcd03ba2881 not found: ID does not exist"
Jul 21 12:49:31.942034 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:31.941991    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-rqktz"]
Jul 21 12:49:31.945500 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:31.945480    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-rqktz"]
Jul 21 12:49:33.194844 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:33.194810    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b0b32ced-e1af-4c19-8f93-e27942b67c37" path="/var/lib/kubelet/pods/b0b32ced-e1af-4c19-8f93-e27942b67c37/volumes"
Jul 21 12:49:38.553452 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:38.553413    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-79f7t"]
Jul 21 12:49:38.553985 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:38.553654    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-79f7t" podUID="5f88a56b-cd09-4b29-8a2e-eb2c8a4ecf9c" containerName="limitador" containerID="cri-o://99d8232341e63a47e94bbf1f18d60c290a779f9cd593012c0423805df6da513f" gracePeriod=30
Jul 21 12:49:38.554446 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:38.554328    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-79f7t"
Jul 21 12:49:39.110408 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:39.110384    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-79f7t"
Jul 21 12:49:39.181810 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:39.181754    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-98g8v\" (UniqueName: \"kubernetes.io/projected/5f88a56b-cd09-4b29-8a2e-eb2c8a4ecf9c-kube-api-access-98g8v\") pod \"5f88a56b-cd09-4b29-8a2e-eb2c8a4ecf9c\" (UID: \"5f88a56b-cd09-4b29-8a2e-eb2c8a4ecf9c\") "
Jul 21 12:49:39.181810 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:39.181798    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/5f88a56b-cd09-4b29-8a2e-eb2c8a4ecf9c-config-file\") pod \"5f88a56b-cd09-4b29-8a2e-eb2c8a4ecf9c\" (UID: \"5f88a56b-cd09-4b29-8a2e-eb2c8a4ecf9c\") "
Jul 21 12:49:39.182173 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:39.182153    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5f88a56b-cd09-4b29-8a2e-eb2c8a4ecf9c-config-file" (OuterVolumeSpecName: "config-file") pod "5f88a56b-cd09-4b29-8a2e-eb2c8a4ecf9c" (UID: "5f88a56b-cd09-4b29-8a2e-eb2c8a4ecf9c"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 12:49:39.183886 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:39.183859    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5f88a56b-cd09-4b29-8a2e-eb2c8a4ecf9c-kube-api-access-98g8v" (OuterVolumeSpecName: "kube-api-access-98g8v") pod "5f88a56b-cd09-4b29-8a2e-eb2c8a4ecf9c" (UID: "5f88a56b-cd09-4b29-8a2e-eb2c8a4ecf9c"). InnerVolumeSpecName "kube-api-access-98g8v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 12:49:39.282867 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:39.282837    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-98g8v\" (UniqueName: \"kubernetes.io/projected/5f88a56b-cd09-4b29-8a2e-eb2c8a4ecf9c-kube-api-access-98g8v\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Jul 21 12:49:39.282867 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:39.282862    2556 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/5f88a56b-cd09-4b29-8a2e-eb2c8a4ecf9c-config-file\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Jul 21 12:49:39.955367 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:39.955340    2556 generic.go:356] "Generic (PLEG): container finished" podID="5f88a56b-cd09-4b29-8a2e-eb2c8a4ecf9c" containerID="99d8232341e63a47e94bbf1f18d60c290a779f9cd593012c0423805df6da513f" exitCode=0
Jul 21 12:49:39.955754 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:39.955382    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-79f7t"
Jul 21 12:49:39.955754 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:39.955434    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-79f7t" event={"ID":"5f88a56b-cd09-4b29-8a2e-eb2c8a4ecf9c","Type":"ContainerDied","Data":"99d8232341e63a47e94bbf1f18d60c290a779f9cd593012c0423805df6da513f"}
Jul 21 12:49:39.955754 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:39.955489    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-79f7t" event={"ID":"5f88a56b-cd09-4b29-8a2e-eb2c8a4ecf9c","Type":"ContainerDied","Data":"5e93dd9b3496ddf66e900e7d45a139147834ee883e063ae8a04f07d1a522b1a8"}
Jul 21 12:49:39.955754 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:39.955514    2556 scope.go:117] "RemoveContainer" containerID="99d8232341e63a47e94bbf1f18d60c290a779f9cd593012c0423805df6da513f"
Jul 21 12:49:39.964725 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:39.964709    2556 scope.go:117] "RemoveContainer" containerID="99d8232341e63a47e94bbf1f18d60c290a779f9cd593012c0423805df6da513f"
Jul 21 12:49:39.964991 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:49:39.964973    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"99d8232341e63a47e94bbf1f18d60c290a779f9cd593012c0423805df6da513f\": container with ID starting with 99d8232341e63a47e94bbf1f18d60c290a779f9cd593012c0423805df6da513f not found: ID does not exist" containerID="99d8232341e63a47e94bbf1f18d60c290a779f9cd593012c0423805df6da513f"
Jul 21 12:49:39.965067 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:39.964999    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"99d8232341e63a47e94bbf1f18d60c290a779f9cd593012c0423805df6da513f"} err="failed to get container status \"99d8232341e63a47e94bbf1f18d60c290a779f9cd593012c0423805df6da513f\": rpc error: code = NotFound desc = could not find container \"99d8232341e63a47e94bbf1f18d60c290a779f9cd593012c0423805df6da513f\": container with ID starting with 99d8232341e63a47e94bbf1f18d60c290a779f9cd593012c0423805df6da513f not found: ID does not exist"
Jul 21 12:49:39.979770 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:39.979745    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-79f7t"]
Jul 21 12:49:39.983891 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:39.983865    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-79f7t"]
Jul 21 12:49:41.194641 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:41.194605    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5f88a56b-cd09-4b29-8a2e-eb2c8a4ecf9c" path="/var/lib/kubelet/pods/5f88a56b-cd09-4b29-8a2e-eb2c8a4ecf9c/volumes"
Jul 21 12:49:44.693355 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:44.693318    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-44bvs"]
Jul 21 12:49:44.693979 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:44.693952    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b0b32ced-e1af-4c19-8f93-e27942b67c37" containerName="authorino"
Jul 21 12:49:44.693979 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:44.693977    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="b0b32ced-e1af-4c19-8f93-e27942b67c37" containerName="authorino"
Jul 21 12:49:44.694246 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:44.694037    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f88a56b-cd09-4b29-8a2e-eb2c8a4ecf9c" containerName="limitador"
Jul 21 12:49:44.694246 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:44.694056    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f88a56b-cd09-4b29-8a2e-eb2c8a4ecf9c" containerName="limitador"
Jul 21 12:49:44.694246 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:44.694177    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="5f88a56b-cd09-4b29-8a2e-eb2c8a4ecf9c" containerName="limitador"
Jul 21 12:49:44.694246 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:44.694194    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="b0b32ced-e1af-4c19-8f93-e27942b67c37" containerName="authorino"
Jul 21 12:49:44.698599 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:44.698581    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-44bvs"
Jul 21 12:49:44.701136 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:44.701100    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 21 12:49:44.701136 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:44.701120    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-d2rpp\""
Jul 21 12:49:44.701374 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:44.701355    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 21 12:49:44.701625 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:44.701606    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Jul 21 12:49:44.707187 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:44.707162    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-44bvs"]
Jul 21 12:49:44.725215 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:44.725195    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/a9693639-fb3b-495e-9669-bf8a0a2b7127-data\") pod \"postgres-69fb4594cf-44bvs\" (UID: \"a9693639-fb3b-495e-9669-bf8a0a2b7127\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-44bvs"
Jul 21 12:49:44.725333 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:44.725303    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v7g72\" (UniqueName: \"kubernetes.io/projected/a9693639-fb3b-495e-9669-bf8a0a2b7127-kube-api-access-v7g72\") pod \"postgres-69fb4594cf-44bvs\" (UID: \"a9693639-fb3b-495e-9669-bf8a0a2b7127\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-44bvs"
Jul 21 12:49:44.825714 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:44.825685    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/a9693639-fb3b-495e-9669-bf8a0a2b7127-data\") pod \"postgres-69fb4594cf-44bvs\" (UID: \"a9693639-fb3b-495e-9669-bf8a0a2b7127\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-44bvs"
Jul 21 12:49:44.825830 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:44.825762    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v7g72\" (UniqueName: \"kubernetes.io/projected/a9693639-fb3b-495e-9669-bf8a0a2b7127-kube-api-access-v7g72\") pod \"postgres-69fb4594cf-44bvs\" (UID: \"a9693639-fb3b-495e-9669-bf8a0a2b7127\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-44bvs"
Jul 21 12:49:44.826091 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:44.826076    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/a9693639-fb3b-495e-9669-bf8a0a2b7127-data\") pod \"postgres-69fb4594cf-44bvs\" (UID: \"a9693639-fb3b-495e-9669-bf8a0a2b7127\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-44bvs"
Jul 21 12:49:44.837294 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:44.837269    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v7g72\" (UniqueName: \"kubernetes.io/projected/a9693639-fb3b-495e-9669-bf8a0a2b7127-kube-api-access-v7g72\") pod \"postgres-69fb4594cf-44bvs\" (UID: \"a9693639-fb3b-495e-9669-bf8a0a2b7127\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-44bvs"
Jul 21 12:49:45.012330 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:45.012309    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-44bvs"
Jul 21 12:49:45.150935 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:45.150912    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-44bvs"]
Jul 21 12:49:45.153448 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:49:45.153426    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9693639_fb3b_495e_9669_bf8a0a2b7127.slice/crio-cbfc6508e851304797e0859724a297ac7cebbb19cc506f7508f45fdc256f9017 WatchSource:0}: Error finding container cbfc6508e851304797e0859724a297ac7cebbb19cc506f7508f45fdc256f9017: Status 404 returned error can't find the container with id cbfc6508e851304797e0859724a297ac7cebbb19cc506f7508f45fdc256f9017
Jul 21 12:49:45.992964 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:45.992903    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-44bvs" event={"ID":"a9693639-fb3b-495e-9669-bf8a0a2b7127","Type":"ContainerStarted","Data":"cbfc6508e851304797e0859724a297ac7cebbb19cc506f7508f45fdc256f9017"}
Jul 21 12:49:49.908598 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:49.908570    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Jul 21 12:49:50.017269 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:50.017237    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-44bvs" event={"ID":"a9693639-fb3b-495e-9669-bf8a0a2b7127","Type":"ContainerStarted","Data":"7169d0b734f63331ab328cf039a161cfa3e36980f17cf074dcaa2a50e38738a2"}
Jul 21 12:49:50.017420 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:50.017400    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-44bvs"
Jul 21 12:49:50.042333 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:50.042283    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-44bvs" podStartSLOduration=1.299706435 podStartE2EDuration="6.042267513s" totalImagesPullingTime="4.742561078s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:49:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:49:50.041161681 +0000 UTC m=+603.345166801" watchObservedRunningTime="2026-07-21 12:49:50.042267513 +0000 UTC m=+603.346272649"
Jul 21 12:49:56.053856 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:56.053819    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-44bvs"
Jul 21 12:49:56.600244 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:56.600216    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-m2s7p"]
Jul 21 12:49:56.608809 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:56.608786    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-m2s7p"
Jul 21 12:49:56.609624 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:56.609599    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-m2s7p"]
Jul 21 12:49:56.611515 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:56.611496    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-m9vxc\""
Jul 21 12:49:56.726669 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:56.726642    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5zdbm\" (UniqueName: \"kubernetes.io/projected/94ae7830-33c4-4299-8437-76e42827e414-kube-api-access-5zdbm\") pod \"authorino-5d4b67c446-m2s7p\" (UID: \"94ae7830-33c4-4299-8437-76e42827e414\") " pod="kuadrant-system/authorino-5d4b67c446-m2s7p"
Jul 21 12:49:56.827531 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:56.827503    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5zdbm\" (UniqueName: \"kubernetes.io/projected/94ae7830-33c4-4299-8437-76e42827e414-kube-api-access-5zdbm\") pod \"authorino-5d4b67c446-m2s7p\" (UID: \"94ae7830-33c4-4299-8437-76e42827e414\") " pod="kuadrant-system/authorino-5d4b67c446-m2s7p"
Jul 21 12:49:56.838307 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:56.838284    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5zdbm\" (UniqueName: \"kubernetes.io/projected/94ae7830-33c4-4299-8437-76e42827e414-kube-api-access-5zdbm\") pod \"authorino-5d4b67c446-m2s7p\" (UID: \"94ae7830-33c4-4299-8437-76e42827e414\") " pod="kuadrant-system/authorino-5d4b67c446-m2s7p"
Jul 21 12:49:56.842525 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:56.842504    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-m2s7p"]
Jul 21 12:49:56.842820 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:56.842696    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-m2s7p"
Jul 21 12:49:56.876517 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:56.876493    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-8566646bc4-q8268"]
Jul 21 12:49:56.881906 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:56.881888    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8566646bc4-q8268"
Jul 21 12:49:56.890884 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:56.890861    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-8566646bc4-q8268"]
Jul 21 12:49:56.928794 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:56.928767    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qzlnd\" (UniqueName: \"kubernetes.io/projected/fad547eb-2bd0-49f8-a769-4ffb554d264c-kube-api-access-qzlnd\") pod \"authorino-8566646bc4-q8268\" (UID: \"fad547eb-2bd0-49f8-a769-4ffb554d264c\") " pod="kuadrant-system/authorino-8566646bc4-q8268"
Jul 21 12:49:56.987643 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:56.987616    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-m2s7p"]
Jul 21 12:49:56.990644 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:49:56.990613    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod94ae7830_33c4_4299_8437_76e42827e414.slice/crio-0ef26ab748c7762b407a7426c5039262a18dd4e8800de5554ad78bb9a63a2fbb WatchSource:0}: Error finding container 0ef26ab748c7762b407a7426c5039262a18dd4e8800de5554ad78bb9a63a2fbb: Status 404 returned error can't find the container with id 0ef26ab748c7762b407a7426c5039262a18dd4e8800de5554ad78bb9a63a2fbb
Jul 21 12:49:57.029920 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:57.029893    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qzlnd\" (UniqueName: \"kubernetes.io/projected/fad547eb-2bd0-49f8-a769-4ffb554d264c-kube-api-access-qzlnd\") pod \"authorino-8566646bc4-q8268\" (UID: \"fad547eb-2bd0-49f8-a769-4ffb554d264c\") " pod="kuadrant-system/authorino-8566646bc4-q8268"
Jul 21 12:49:57.040190 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:57.040159    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qzlnd\" (UniqueName: \"kubernetes.io/projected/fad547eb-2bd0-49f8-a769-4ffb554d264c-kube-api-access-qzlnd\") pod \"authorino-8566646bc4-q8268\" (UID: \"fad547eb-2bd0-49f8-a769-4ffb554d264c\") " pod="kuadrant-system/authorino-8566646bc4-q8268"
Jul 21 12:49:57.054890 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:57.054864    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-m2s7p" event={"ID":"94ae7830-33c4-4299-8437-76e42827e414","Type":"ContainerStarted","Data":"0ef26ab748c7762b407a7426c5039262a18dd4e8800de5554ad78bb9a63a2fbb"}
Jul 21 12:49:57.108506 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:57.108453    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-8566646bc4-q8268"]
Jul 21 12:49:57.108708 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:57.108694    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8566646bc4-q8268"
Jul 21 12:49:57.139867 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:57.139842    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7bb9f58598-mssv5"]
Jul 21 12:49:57.144957 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:57.144935    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bb9f58598-mssv5"
Jul 21 12:49:57.147976 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:57.147955    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Jul 21 12:49:57.155542 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:57.155519    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7bb9f58598-mssv5"]
Jul 21 12:49:57.232544 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:57.232517    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/2b659e93-adc5-433a-9074-fd4a5b285e74-tls-cert\") pod \"authorino-7bb9f58598-mssv5\" (UID: \"2b659e93-adc5-433a-9074-fd4a5b285e74\") " pod="kuadrant-system/authorino-7bb9f58598-mssv5"
Jul 21 12:49:57.232685 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:57.232634    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s6nxt\" (UniqueName: \"kubernetes.io/projected/2b659e93-adc5-433a-9074-fd4a5b285e74-kube-api-access-s6nxt\") pod \"authorino-7bb9f58598-mssv5\" (UID: \"2b659e93-adc5-433a-9074-fd4a5b285e74\") " pod="kuadrant-system/authorino-7bb9f58598-mssv5"
Jul 21 12:49:57.249977 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:57.249955    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-8566646bc4-q8268"]
Jul 21 12:49:57.251259 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:49:57.251238    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfad547eb_2bd0_49f8_a769_4ffb554d264c.slice/crio-a8fd3a3dd2a1c4b3607bd79b5cf91337c78c4b14becdf7f683b735da6651fd23 WatchSource:0}: Error finding container a8fd3a3dd2a1c4b3607bd79b5cf91337c78c4b14becdf7f683b735da6651fd23: Status 404 returned error can't find the container with id a8fd3a3dd2a1c4b3607bd79b5cf91337c78c4b14becdf7f683b735da6651fd23
Jul 21 12:49:57.333769 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:57.333738    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s6nxt\" (UniqueName: \"kubernetes.io/projected/2b659e93-adc5-433a-9074-fd4a5b285e74-kube-api-access-s6nxt\") pod \"authorino-7bb9f58598-mssv5\" (UID: \"2b659e93-adc5-433a-9074-fd4a5b285e74\") " pod="kuadrant-system/authorino-7bb9f58598-mssv5"
Jul 21 12:49:57.333921 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:57.333821    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/2b659e93-adc5-433a-9074-fd4a5b285e74-tls-cert\") pod \"authorino-7bb9f58598-mssv5\" (UID: \"2b659e93-adc5-433a-9074-fd4a5b285e74\") " pod="kuadrant-system/authorino-7bb9f58598-mssv5"
Jul 21 12:49:57.336235 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:57.336214    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/2b659e93-adc5-433a-9074-fd4a5b285e74-tls-cert\") pod \"authorino-7bb9f58598-mssv5\" (UID: \"2b659e93-adc5-433a-9074-fd4a5b285e74\") " pod="kuadrant-system/authorino-7bb9f58598-mssv5"
Jul 21 12:49:57.348802 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:57.348781    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s6nxt\" (UniqueName: \"kubernetes.io/projected/2b659e93-adc5-433a-9074-fd4a5b285e74-kube-api-access-s6nxt\") pod \"authorino-7bb9f58598-mssv5\" (UID: \"2b659e93-adc5-433a-9074-fd4a5b285e74\") " pod="kuadrant-system/authorino-7bb9f58598-mssv5"
Jul 21 12:49:57.457673 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:57.457622    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bb9f58598-mssv5"
Jul 21 12:49:57.597643 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:57.597619    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7bb9f58598-mssv5"]
Jul 21 12:49:57.599224 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:49:57.599202    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2b659e93_adc5_433a_9074_fd4a5b285e74.slice/crio-e126d99239819ea3557d2ac00e4fe72fa744b3b5817f59378afd11a8cb464cb0 WatchSource:0}: Error finding container e126d99239819ea3557d2ac00e4fe72fa744b3b5817f59378afd11a8cb464cb0: Status 404 returned error can't find the container with id e126d99239819ea3557d2ac00e4fe72fa744b3b5817f59378afd11a8cb464cb0
Jul 21 12:49:58.062150 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:58.062124    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-m2s7p" event={"ID":"94ae7830-33c4-4299-8437-76e42827e414","Type":"ContainerStarted","Data":"219a9b53ec4dd09655fe73e40a700bcb45e502dc942ef92343226c94d7bf7ed3"}
Jul 21 12:49:58.062465 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:58.062186    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-5d4b67c446-m2s7p" podUID="94ae7830-33c4-4299-8437-76e42827e414" containerName="authorino" containerID="cri-o://219a9b53ec4dd09655fe73e40a700bcb45e502dc942ef92343226c94d7bf7ed3" gracePeriod=30
Jul 21 12:49:58.063900 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:58.063874    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bb9f58598-mssv5" event={"ID":"2b659e93-adc5-433a-9074-fd4a5b285e74","Type":"ContainerStarted","Data":"0a1c67e1c9153154dac17c12146992a5b369805156ee430d2561dbfdbcc71f1e"}
Jul 21 12:49:58.064007 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:58.063916    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bb9f58598-mssv5" event={"ID":"2b659e93-adc5-433a-9074-fd4a5b285e74","Type":"ContainerStarted","Data":"e126d99239819ea3557d2ac00e4fe72fa744b3b5817f59378afd11a8cb464cb0"}
Jul 21 12:49:58.065732 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:58.065707    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-8566646bc4-q8268" event={"ID":"fad547eb-2bd0-49f8-a769-4ffb554d264c","Type":"ContainerStarted","Data":"ffa65f3b08a494e38871ff3285e3819c084e5c115ed8cab7927b35d11e1a5238"}
Jul 21 12:49:58.065843 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:58.065745    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-8566646bc4-q8268" event={"ID":"fad547eb-2bd0-49f8-a769-4ffb554d264c","Type":"ContainerStarted","Data":"a8fd3a3dd2a1c4b3607bd79b5cf91337c78c4b14becdf7f683b735da6651fd23"}
Jul 21 12:49:58.065914 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:58.065839    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-8566646bc4-q8268" podUID="fad547eb-2bd0-49f8-a769-4ffb554d264c" containerName="authorino" containerID="cri-o://ffa65f3b08a494e38871ff3285e3819c084e5c115ed8cab7927b35d11e1a5238" gracePeriod=30
Jul 21 12:49:58.078771 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:58.078729    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d4b67c446-m2s7p" podStartSLOduration=1.432935957 podStartE2EDuration="2.078712683s" totalImagesPullingTime="645.776726ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:49:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:49:58.078249654 +0000 UTC m=+611.382254749" watchObservedRunningTime="2026-07-21 12:49:58.078712683 +0000 UTC m=+611.382717776"
Jul 21 12:49:58.098439 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:58.098400    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-8566646bc4-q8268" podStartSLOduration=1.7548263039999998 podStartE2EDuration="2.09838118s" totalImagesPullingTime="343.554876ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:49:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:49:58.094679147 +0000 UTC m=+611.398684237" watchObservedRunningTime="2026-07-21 12:49:58.09838118 +0000 UTC m=+611.402386271"
Jul 21 12:49:58.115048 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:58.114910    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7bb9f58598-mssv5" podStartSLOduration=0.737533175 podStartE2EDuration="1.114891374s" totalImagesPullingTime="377.358199ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:49:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:49:58.114146647 +0000 UTC m=+611.418151739" watchObservedRunningTime="2026-07-21 12:49:58.114891374 +0000 UTC m=+611.418896465"
Jul 21 12:49:58.330055 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:58.330032    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-m2s7p"
Jul 21 12:49:58.333151 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:58.333133    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8566646bc4-q8268"
Jul 21 12:49:58.445887 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:58.445857    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qzlnd\" (UniqueName: \"kubernetes.io/projected/fad547eb-2bd0-49f8-a769-4ffb554d264c-kube-api-access-qzlnd\") pod \"fad547eb-2bd0-49f8-a769-4ffb554d264c\" (UID: \"fad547eb-2bd0-49f8-a769-4ffb554d264c\") "
Jul 21 12:49:58.446063 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:58.445892    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5zdbm\" (UniqueName: \"kubernetes.io/projected/94ae7830-33c4-4299-8437-76e42827e414-kube-api-access-5zdbm\") pod \"94ae7830-33c4-4299-8437-76e42827e414\" (UID: \"94ae7830-33c4-4299-8437-76e42827e414\") "
Jul 21 12:49:58.448119 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:58.448093    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/94ae7830-33c4-4299-8437-76e42827e414-kube-api-access-5zdbm" (OuterVolumeSpecName: "kube-api-access-5zdbm") pod "94ae7830-33c4-4299-8437-76e42827e414" (UID: "94ae7830-33c4-4299-8437-76e42827e414"). InnerVolumeSpecName "kube-api-access-5zdbm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 12:49:58.448203 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:58.448118    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fad547eb-2bd0-49f8-a769-4ffb554d264c-kube-api-access-qzlnd" (OuterVolumeSpecName: "kube-api-access-qzlnd") pod "fad547eb-2bd0-49f8-a769-4ffb554d264c" (UID: "fad547eb-2bd0-49f8-a769-4ffb554d264c"). InnerVolumeSpecName "kube-api-access-qzlnd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 12:49:58.546863 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:58.546806    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qzlnd\" (UniqueName: \"kubernetes.io/projected/fad547eb-2bd0-49f8-a769-4ffb554d264c-kube-api-access-qzlnd\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Jul 21 12:49:58.546863 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:58.546826    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5zdbm\" (UniqueName: \"kubernetes.io/projected/94ae7830-33c4-4299-8437-76e42827e414-kube-api-access-5zdbm\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Jul 21 12:49:59.072149 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:59.072118    2556 generic.go:356] "Generic (PLEG): container finished" podID="fad547eb-2bd0-49f8-a769-4ffb554d264c" containerID="ffa65f3b08a494e38871ff3285e3819c084e5c115ed8cab7927b35d11e1a5238" exitCode=0
Jul 21 12:49:59.072552 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:59.072162    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8566646bc4-q8268"
Jul 21 12:49:59.072552 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:59.072204    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-8566646bc4-q8268" event={"ID":"fad547eb-2bd0-49f8-a769-4ffb554d264c","Type":"ContainerDied","Data":"ffa65f3b08a494e38871ff3285e3819c084e5c115ed8cab7927b35d11e1a5238"}
Jul 21 12:49:59.072552 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:59.072241    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-8566646bc4-q8268" event={"ID":"fad547eb-2bd0-49f8-a769-4ffb554d264c","Type":"ContainerDied","Data":"a8fd3a3dd2a1c4b3607bd79b5cf91337c78c4b14becdf7f683b735da6651fd23"}
Jul 21 12:49:59.072552 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:59.072262    2556 scope.go:117] "RemoveContainer" containerID="ffa65f3b08a494e38871ff3285e3819c084e5c115ed8cab7927b35d11e1a5238"
Jul 21 12:49:59.073549 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:59.073516    2556 generic.go:356] "Generic (PLEG): container finished" podID="94ae7830-33c4-4299-8437-76e42827e414" containerID="219a9b53ec4dd09655fe73e40a700bcb45e502dc942ef92343226c94d7bf7ed3" exitCode=0
Jul 21 12:49:59.073639 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:59.073555    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-m2s7p"
Jul 21 12:49:59.073639 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:59.073597    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-m2s7p" event={"ID":"94ae7830-33c4-4299-8437-76e42827e414","Type":"ContainerDied","Data":"219a9b53ec4dd09655fe73e40a700bcb45e502dc942ef92343226c94d7bf7ed3"}
Jul 21 12:49:59.073639 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:59.073626    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-m2s7p" event={"ID":"94ae7830-33c4-4299-8437-76e42827e414","Type":"ContainerDied","Data":"0ef26ab748c7762b407a7426c5039262a18dd4e8800de5554ad78bb9a63a2fbb"}
Jul 21 12:49:59.081275 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:59.080935    2556 scope.go:117] "RemoveContainer" containerID="ffa65f3b08a494e38871ff3285e3819c084e5c115ed8cab7927b35d11e1a5238"
Jul 21 12:49:59.081373 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:49:59.081272    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ffa65f3b08a494e38871ff3285e3819c084e5c115ed8cab7927b35d11e1a5238\": container with ID starting with ffa65f3b08a494e38871ff3285e3819c084e5c115ed8cab7927b35d11e1a5238 not found: ID does not exist" containerID="ffa65f3b08a494e38871ff3285e3819c084e5c115ed8cab7927b35d11e1a5238"
Jul 21 12:49:59.081373 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:59.081304    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ffa65f3b08a494e38871ff3285e3819c084e5c115ed8cab7927b35d11e1a5238"} err="failed to get container status \"ffa65f3b08a494e38871ff3285e3819c084e5c115ed8cab7927b35d11e1a5238\": rpc error: code = NotFound desc = could not find container \"ffa65f3b08a494e38871ff3285e3819c084e5c115ed8cab7927b35d11e1a5238\": container with ID starting with ffa65f3b08a494e38871ff3285e3819c084e5c115ed8cab7927b35d11e1a5238 not found: ID does not exist"
Jul 21 12:49:59.081373 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:59.081328    2556 scope.go:117] "RemoveContainer" containerID="219a9b53ec4dd09655fe73e40a700bcb45e502dc942ef92343226c94d7bf7ed3"
Jul 21 12:49:59.089798 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:59.089779    2556 scope.go:117] "RemoveContainer" containerID="219a9b53ec4dd09655fe73e40a700bcb45e502dc942ef92343226c94d7bf7ed3"
Jul 21 12:49:59.090062 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:49:59.090044    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"219a9b53ec4dd09655fe73e40a700bcb45e502dc942ef92343226c94d7bf7ed3\": container with ID starting with 219a9b53ec4dd09655fe73e40a700bcb45e502dc942ef92343226c94d7bf7ed3 not found: ID does not exist" containerID="219a9b53ec4dd09655fe73e40a700bcb45e502dc942ef92343226c94d7bf7ed3"
Jul 21 12:49:59.090135 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:59.090069    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"219a9b53ec4dd09655fe73e40a700bcb45e502dc942ef92343226c94d7bf7ed3"} err="failed to get container status \"219a9b53ec4dd09655fe73e40a700bcb45e502dc942ef92343226c94d7bf7ed3\": rpc error: code = NotFound desc = could not find container \"219a9b53ec4dd09655fe73e40a700bcb45e502dc942ef92343226c94d7bf7ed3\": container with ID starting with 219a9b53ec4dd09655fe73e40a700bcb45e502dc942ef92343226c94d7bf7ed3 not found: ID does not exist"
Jul 21 12:49:59.116198 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:59.116151    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-m2s7p"]
Jul 21 12:49:59.118880 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:59.118863    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-m2s7p"]
Jul 21 12:49:59.145735 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:59.145716    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-8566646bc4-q8268"]
Jul 21 12:49:59.149916 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:59.149897    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-8566646bc4-q8268"]
Jul 21 12:49:59.194220 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:59.194198    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="94ae7830-33c4-4299-8437-76e42827e414" path="/var/lib/kubelet/pods/94ae7830-33c4-4299-8437-76e42827e414/volumes"
Jul 21 12:49:59.194545 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:49:59.194532    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fad547eb-2bd0-49f8-a769-4ffb554d264c" path="/var/lib/kubelet/pods/fad547eb-2bd0-49f8-a769-4ffb554d264c/volumes"
Jul 21 12:50:35.548789 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:50:35.548758    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-cf4f9bc89-zsldg"]
Jul 21 12:50:35.551437 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:50:35.549169    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fad547eb-2bd0-49f8-a769-4ffb554d264c" containerName="authorino"
Jul 21 12:50:35.551437 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:50:35.549181    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="fad547eb-2bd0-49f8-a769-4ffb554d264c" containerName="authorino"
Jul 21 12:50:35.551437 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:50:35.549202    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="94ae7830-33c4-4299-8437-76e42827e414" containerName="authorino"
Jul 21 12:50:35.551437 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:50:35.549207    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="94ae7830-33c4-4299-8437-76e42827e414" containerName="authorino"
Jul 21 12:50:35.551437 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:50:35.549264    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="fad547eb-2bd0-49f8-a769-4ffb554d264c" containerName="authorino"
Jul 21 12:50:35.551437 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:50:35.549276    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="94ae7830-33c4-4299-8437-76e42827e414" containerName="authorino"
Jul 21 12:50:35.552327 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:50:35.552313    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-cf4f9bc89-zsldg"
Jul 21 12:50:35.554878 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:50:35.554846    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Jul 21 12:50:35.554878 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:50:35.554875    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-njpqk\""
Jul 21 12:50:35.561226 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:50:35.561204    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-cf4f9bc89-zsldg"]
Jul 21 12:50:35.649072 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:50:35.649035    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/621e7dbe-d55a-4e01-94e7-a30afdf644d6-maas-api-tls\") pod \"maas-api-cf4f9bc89-zsldg\" (UID: \"621e7dbe-d55a-4e01-94e7-a30afdf644d6\") " pod="odh-ai-gateway-infra/maas-api-cf4f9bc89-zsldg"
Jul 21 12:50:35.649223 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:50:35.649182    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f9hkw\" (UniqueName: \"kubernetes.io/projected/621e7dbe-d55a-4e01-94e7-a30afdf644d6-kube-api-access-f9hkw\") pod \"maas-api-cf4f9bc89-zsldg\" (UID: \"621e7dbe-d55a-4e01-94e7-a30afdf644d6\") " pod="odh-ai-gateway-infra/maas-api-cf4f9bc89-zsldg"
Jul 21 12:50:35.649223 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:50:35.649216    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/621e7dbe-d55a-4e01-94e7-a30afdf644d6-openshift-service-ca\") pod \"maas-api-cf4f9bc89-zsldg\" (UID: \"621e7dbe-d55a-4e01-94e7-a30afdf644d6\") " pod="odh-ai-gateway-infra/maas-api-cf4f9bc89-zsldg"
Jul 21 12:50:35.750552 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:50:35.750527    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f9hkw\" (UniqueName: \"kubernetes.io/projected/621e7dbe-d55a-4e01-94e7-a30afdf644d6-kube-api-access-f9hkw\") pod \"maas-api-cf4f9bc89-zsldg\" (UID: \"621e7dbe-d55a-4e01-94e7-a30afdf644d6\") " pod="odh-ai-gateway-infra/maas-api-cf4f9bc89-zsldg"
Jul 21 12:50:35.750669 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:50:35.750557    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/621e7dbe-d55a-4e01-94e7-a30afdf644d6-openshift-service-ca\") pod \"maas-api-cf4f9bc89-zsldg\" (UID: \"621e7dbe-d55a-4e01-94e7-a30afdf644d6\") " pod="odh-ai-gateway-infra/maas-api-cf4f9bc89-zsldg"
Jul 21 12:50:35.750669 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:50:35.750612    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/621e7dbe-d55a-4e01-94e7-a30afdf644d6-maas-api-tls\") pod \"maas-api-cf4f9bc89-zsldg\" (UID: \"621e7dbe-d55a-4e01-94e7-a30afdf644d6\") " pod="odh-ai-gateway-infra/maas-api-cf4f9bc89-zsldg"
Jul 21 12:50:35.751193 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:50:35.751173    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/621e7dbe-d55a-4e01-94e7-a30afdf644d6-openshift-service-ca\") pod \"maas-api-cf4f9bc89-zsldg\" (UID: \"621e7dbe-d55a-4e01-94e7-a30afdf644d6\") " pod="odh-ai-gateway-infra/maas-api-cf4f9bc89-zsldg"
Jul 21 12:50:35.753179 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:50:35.753161    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/621e7dbe-d55a-4e01-94e7-a30afdf644d6-maas-api-tls\") pod \"maas-api-cf4f9bc89-zsldg\" (UID: \"621e7dbe-d55a-4e01-94e7-a30afdf644d6\") " pod="odh-ai-gateway-infra/maas-api-cf4f9bc89-zsldg"
Jul 21 12:50:35.762094 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:50:35.762070    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f9hkw\" (UniqueName: \"kubernetes.io/projected/621e7dbe-d55a-4e01-94e7-a30afdf644d6-kube-api-access-f9hkw\") pod \"maas-api-cf4f9bc89-zsldg\" (UID: \"621e7dbe-d55a-4e01-94e7-a30afdf644d6\") " pod="odh-ai-gateway-infra/maas-api-cf4f9bc89-zsldg"
Jul 21 12:50:35.862894 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:50:35.862831    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-cf4f9bc89-zsldg"
Jul 21 12:50:36.006861 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:50:36.006836    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-cf4f9bc89-zsldg"]
Jul 21 12:50:36.008976 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:50:36.008954    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod621e7dbe_d55a_4e01_94e7_a30afdf644d6.slice/crio-e8fc05e9b3b50df42ebe3a2ce13f4276988a8431c22fa98c9476c549d9930edc WatchSource:0}: Error finding container e8fc05e9b3b50df42ebe3a2ce13f4276988a8431c22fa98c9476c549d9930edc: Status 404 returned error can't find the container with id e8fc05e9b3b50df42ebe3a2ce13f4276988a8431c22fa98c9476c549d9930edc
Jul 21 12:50:36.069418 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:50:36.069391    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7677b774bb-xmqdx"]
Jul 21 12:50:36.074309 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:50:36.074291    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7677b774bb-xmqdx"
Jul 21 12:50:36.076711 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:50:36.076689    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Jul 21 12:50:36.076794 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:50:36.076713    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-vffh4\""
Jul 21 12:50:36.083742 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:50:36.083717    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7677b774bb-xmqdx"]
Jul 21 12:50:36.096196 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:50:36.096177    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-q5zgh"]
Jul 21 12:50:36.099693 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:50:36.099675    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5bf47cd988-q5zgh"
Jul 21 12:50:36.111064 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:50:36.111005    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-q5zgh"]
Jul 21 12:50:36.155335 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:50:36.155273    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ed44480e-4c2e-4539-8f87-d6af31005e3d-plugins-config-volume\") pod \"payload-pre-processing-7677b774bb-xmqdx\" (UID: \"ed44480e-4c2e-4539-8f87-d6af31005e3d\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-xmqdx"
Jul 21 12:50:36.155335 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:50:36.155301    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0e9d6e83-1286-4634-866a-3ffcdcb84dbc-plugins-config-volume\") pod \"payload-processing-5bf47cd988-q5zgh\" (UID: \"0e9d6e83-1286-4634-866a-3ffcdcb84dbc\") " pod="openshift-ingress/payload-processing-5bf47cd988-q5zgh"
Jul 21 12:50:36.155462 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:50:36.155347    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6thzz\" (UniqueName: \"kubernetes.io/projected/0e9d6e83-1286-4634-866a-3ffcdcb84dbc-kube-api-access-6thzz\") pod \"payload-processing-5bf47cd988-q5zgh\" (UID: \"0e9d6e83-1286-4634-866a-3ffcdcb84dbc\") " pod="openshift-ingress/payload-processing-5bf47cd988-q5zgh"
Jul 21 12:50:36.155462 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:50:36.155403    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r478j\" (UniqueName: \"kubernetes.io/projected/ed44480e-4c2e-4539-8f87-d6af31005e3d-kube-api-access-r478j\") pod \"payload-pre-processing-7677b774bb-xmqdx\" (UID: \"ed44480e-4c2e-4539-8f87-d6af31005e3d\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-xmqdx"
Jul 21 12:50:36.256117 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:50:36.256086    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6thzz\" (UniqueName: \"kubernetes.io/projected/0e9d6e83-1286-4634-866a-3ffcdcb84dbc-kube-api-access-6thzz\") pod \"payload-processing-5bf47cd988-q5zgh\" (UID: \"0e9d6e83-1286-4634-866a-3ffcdcb84dbc\") " pod="openshift-ingress/payload-processing-5bf47cd988-q5zgh"
Jul 21 12:50:36.256238 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:50:36.256176    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r478j\" (UniqueName: \"kubernetes.io/projected/ed44480e-4c2e-4539-8f87-d6af31005e3d-kube-api-access-r478j\") pod \"payload-pre-processing-7677b774bb-xmqdx\" (UID: \"ed44480e-4c2e-4539-8f87-d6af31005e3d\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-xmqdx"
Jul 21 12:50:36.256297 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:50:36.256268    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ed44480e-4c2e-4539-8f87-d6af31005e3d-plugins-config-volume\") pod \"payload-pre-processing-7677b774bb-xmqdx\" (UID: \"ed44480e-4c2e-4539-8f87-d6af31005e3d\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-xmqdx"
Jul 21 12:50:36.256337 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:50:36.256296    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0e9d6e83-1286-4634-866a-3ffcdcb84dbc-plugins-config-volume\") pod \"payload-processing-5bf47cd988-q5zgh\" (UID: \"0e9d6e83-1286-4634-866a-3ffcdcb84dbc\") " pod="openshift-ingress/payload-processing-5bf47cd988-q5zgh"
Jul 21 12:50:36.256900 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:50:36.256881    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ed44480e-4c2e-4539-8f87-d6af31005e3d-plugins-config-volume\") pod \"payload-pre-processing-7677b774bb-xmqdx\" (UID: \"ed44480e-4c2e-4539-8f87-d6af31005e3d\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-xmqdx"
Jul 21 12:50:36.256979 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:50:36.256925    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0e9d6e83-1286-4634-866a-3ffcdcb84dbc-plugins-config-volume\") pod \"payload-processing-5bf47cd988-q5zgh\" (UID: \"0e9d6e83-1286-4634-866a-3ffcdcb84dbc\") " pod="openshift-ingress/payload-processing-5bf47cd988-q5zgh"
Jul 21 12:50:36.266240 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:50:36.266216    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6thzz\" (UniqueName: \"kubernetes.io/projected/0e9d6e83-1286-4634-866a-3ffcdcb84dbc-kube-api-access-6thzz\") pod \"payload-processing-5bf47cd988-q5zgh\" (UID: \"0e9d6e83-1286-4634-866a-3ffcdcb84dbc\") " pod="openshift-ingress/payload-processing-5bf47cd988-q5zgh"
Jul 21 12:50:36.266895 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:50:36.266873    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r478j\" (UniqueName: \"kubernetes.io/projected/ed44480e-4c2e-4539-8f87-d6af31005e3d-kube-api-access-r478j\") pod \"payload-pre-processing-7677b774bb-xmqdx\" (UID: \"ed44480e-4c2e-4539-8f87-d6af31005e3d\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-xmqdx"
Jul 21 12:50:36.283368 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:50:36.283332    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-cf4f9bc89-zsldg" event={"ID":"621e7dbe-d55a-4e01-94e7-a30afdf644d6","Type":"ContainerStarted","Data":"e8fc05e9b3b50df42ebe3a2ce13f4276988a8431c22fa98c9476c549d9930edc"}
Jul 21 12:50:36.385610 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:50:36.385588    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7677b774bb-xmqdx"
Jul 21 12:50:36.409803 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:50:36.409746    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5bf47cd988-q5zgh"
Jul 21 12:50:36.557183 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:50:36.557162    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-q5zgh"]
Jul 21 12:50:36.559267 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:50:36.559239    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e9d6e83_1286_4634_866a_3ffcdcb84dbc.slice/crio-40c84daf3bb893d86f811057c6ee3c16b343a46092fdac7a5e893d66f912cce9 WatchSource:0}: Error finding container 40c84daf3bb893d86f811057c6ee3c16b343a46092fdac7a5e893d66f912cce9: Status 404 returned error can't find the container with id 40c84daf3bb893d86f811057c6ee3c16b343a46092fdac7a5e893d66f912cce9
Jul 21 12:50:36.747293 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:50:36.747269    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7677b774bb-xmqdx"]
Jul 21 12:50:36.748909 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:50:36.748870    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded44480e_4c2e_4539_8f87_d6af31005e3d.slice/crio-f67eed912784041660319f46b606cdeba55acd071d0501cdfcc5efbfeb875d4e WatchSource:0}: Error finding container f67eed912784041660319f46b606cdeba55acd071d0501cdfcc5efbfeb875d4e: Status 404 returned error can't find the container with id f67eed912784041660319f46b606cdeba55acd071d0501cdfcc5efbfeb875d4e
Jul 21 12:50:37.319732 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:50:37.319135    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7677b774bb-xmqdx" event={"ID":"ed44480e-4c2e-4539-8f87-d6af31005e3d","Type":"ContainerStarted","Data":"f67eed912784041660319f46b606cdeba55acd071d0501cdfcc5efbfeb875d4e"}
Jul 21 12:50:37.324601 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:50:37.324521    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5bf47cd988-q5zgh" event={"ID":"0e9d6e83-1286-4634-866a-3ffcdcb84dbc","Type":"ContainerStarted","Data":"40c84daf3bb893d86f811057c6ee3c16b343a46092fdac7a5e893d66f912cce9"}
Jul 21 12:50:40.345706 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:50:40.345671    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-cf4f9bc89-zsldg" event={"ID":"621e7dbe-d55a-4e01-94e7-a30afdf644d6","Type":"ContainerStarted","Data":"1cb06e701a96dcc62ecfe64f2d9190f719d8b813bda9173c4ae55cda5d584ec1"}
Jul 21 12:50:40.346190 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:50:40.346004    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-cf4f9bc89-zsldg"
Jul 21 12:50:40.347450 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:50:40.347428    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7677b774bb-xmqdx" event={"ID":"ed44480e-4c2e-4539-8f87-d6af31005e3d","Type":"ContainerStarted","Data":"34361c8ec561f5765d8320e0aaaa8247153e3d3fd30b1f963217c7361bc155db"}
Jul 21 12:50:40.347566 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:50:40.347484    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7677b774bb-xmqdx"
Jul 21 12:50:40.348913 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:50:40.348890    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5bf47cd988-q5zgh" event={"ID":"0e9d6e83-1286-4634-866a-3ffcdcb84dbc","Type":"ContainerStarted","Data":"b40b4fc39373023439d58a13bd1e3e902ac3a814b9552f70abd50714dc827011"}
Jul 21 12:50:40.349051 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:50:40.349036    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5bf47cd988-q5zgh"
Jul 21 12:50:40.367126 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:50:40.367067    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-cf4f9bc89-zsldg" podStartSLOduration=1.419071925 podStartE2EDuration="5.367055441s" totalImagesPullingTime="3.947983516s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:50:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:50:40.366090973 +0000 UTC m=+653.670096082" watchObservedRunningTime="2026-07-21 12:50:40.367055441 +0000 UTC m=+653.671060529"
Jul 21 12:50:40.384091 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:50:40.384057    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7677b774bb-xmqdx" podStartSLOduration=1.169969297 podStartE2EDuration="4.384044694s" totalImagesPullingTime="3.214075397s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:50:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:50:40.382700107 +0000 UTC m=+653.686705198" watchObservedRunningTime="2026-07-21 12:50:40.384044694 +0000 UTC m=+653.688049783"
Jul 21 12:50:40.403538 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:50:40.403486    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5bf47cd988-q5zgh" podStartSLOduration=1.006278829 podStartE2EDuration="4.403468757s" totalImagesPullingTime="3.397189928s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:50:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:50:40.400879752 +0000 UTC m=+653.704884875" watchObservedRunningTime="2026-07-21 12:50:40.403468757 +0000 UTC m=+653.707473851"
Jul 21 12:50:44.372136 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:50:44.372105    2556 generic.go:356] "Generic (PLEG): container finished" podID="fe4ed9c9-5046-4bb2-acbd-8d83d8b8eb2c" containerID="3244c07f82a77cafd767cfa4e6a844ac4492dae67ff128ebe549bbf7fcf6ef67" exitCode=0
Jul 21 12:50:44.372474 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:50:44.372174    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-b55kr" event={"ID":"fe4ed9c9-5046-4bb2-acbd-8d83d8b8eb2c","Type":"ContainerDied","Data":"3244c07f82a77cafd767cfa4e6a844ac4492dae67ff128ebe549bbf7fcf6ef67"}
Jul 21 12:50:44.372528 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:50:44.372515    2556 scope.go:117] "RemoveContainer" containerID="3244c07f82a77cafd767cfa4e6a844ac4492dae67ff128ebe549bbf7fcf6ef67"
Jul 21 12:50:44.373523 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:50:44.373497    2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 21 12:50:44.373598 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:50:44.373580    2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 21 12:50:44.373669 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:50:44.373652    2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 21 12:50:45.379087 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:50:45.379057    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-b55kr" event={"ID":"fe4ed9c9-5046-4bb2-acbd-8d83d8b8eb2c","Type":"ContainerStarted","Data":"aaecab1430d5e94ace80d6c28b88876d652e0982bad24324317cea85f5f5759d"}
Jul 21 12:50:45.525658 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:50:45.525623    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-b55kr"
Jul 21 12:50:45.525828 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:50:45.525802    2556 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-b55kr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body=
Jul 21 12:50:45.525881 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:50:45.525845    2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-b55kr" podUID="fe4ed9c9-5046-4bb2-acbd-8d83d8b8eb2c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused"
Jul 21 12:50:46.359852 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:50:46.359825    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-cf4f9bc89-zsldg"
Jul 21 12:50:46.525542 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:50:46.525506    2556 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-b55kr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body=
Jul 21 12:50:46.525931 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:50:46.525563    2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-b55kr" podUID="fe4ed9c9-5046-4bb2-acbd-8d83d8b8eb2c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused"
Jul 21 12:50:47.525679 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:50:47.525647    2556 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-b55kr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body=
Jul 21 12:50:47.526108 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:50:47.525735    2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-b55kr" podUID="fe4ed9c9-5046-4bb2-acbd-8d83d8b8eb2c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused"
Jul 21 12:50:48.525810 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:50:48.525776    2556 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-b55kr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body=
Jul 21 12:50:48.526252 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:50:48.525826    2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-b55kr" podUID="fe4ed9c9-5046-4bb2-acbd-8d83d8b8eb2c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused"
Jul 21 12:50:49.529281 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:50:49.529257    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-b55kr"
Jul 21 12:50:49.529725 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:50:49.529711    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-b55kr"
Jul 21 12:50:49.530362 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:50:49.530346    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-b55kr"
Jul 21 12:50:51.356674 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:50:51.356641    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7677b774bb-xmqdx"
Jul 21 12:50:51.357060 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:50:51.356699    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5bf47cd988-q5zgh"
Jul 21 12:51:02.456813 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:51:02.456786    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-ng878"]
Jul 21 12:51:02.463930 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:51:02.463909    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ng878"
Jul 21 12:51:02.466263 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:51:02.466242    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-gbts6\""
Jul 21 12:51:02.467038 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:51:02.467000    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 21 12:51:02.467146 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:51:02.467121    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Jul 21 12:51:02.467264 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:51:02.467244    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 21 12:51:02.469781 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:51:02.469762    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-ng878"]
Jul 21 12:51:02.619409 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:51:02.619375    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/067dd5c2-04fa-471c-bdd3-652ea5c57333-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-ng878\" (UID: \"067dd5c2-04fa-471c-bdd3-652ea5c57333\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ng878"
Jul 21 12:51:02.619409 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:51:02.619405    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/067dd5c2-04fa-471c-bdd3-652ea5c57333-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-ng878\" (UID: \"067dd5c2-04fa-471c-bdd3-652ea5c57333\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ng878"
Jul 21 12:51:02.619592 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:51:02.619424    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/067dd5c2-04fa-471c-bdd3-652ea5c57333-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-ng878\" (UID: \"067dd5c2-04fa-471c-bdd3-652ea5c57333\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ng878"
Jul 21 12:51:02.619592 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:51:02.619474    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/067dd5c2-04fa-471c-bdd3-652ea5c57333-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-ng878\" (UID: \"067dd5c2-04fa-471c-bdd3-652ea5c57333\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ng878"
Jul 21 12:51:02.619592 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:51:02.619520    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h5jbq\" (UniqueName: \"kubernetes.io/projected/067dd5c2-04fa-471c-bdd3-652ea5c57333-kube-api-access-h5jbq\") pod \"e2e-distinct-simulated-kserve-59975d69f9-ng878\" (UID: \"067dd5c2-04fa-471c-bdd3-652ea5c57333\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ng878"
Jul 21 12:51:02.619592 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:51:02.619553    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/067dd5c2-04fa-471c-bdd3-652ea5c57333-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-ng878\" (UID: \"067dd5c2-04fa-471c-bdd3-652ea5c57333\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ng878"
Jul 21 12:51:02.720578 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:51:02.720508    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/067dd5c2-04fa-471c-bdd3-652ea5c57333-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-ng878\" (UID: \"067dd5c2-04fa-471c-bdd3-652ea5c57333\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ng878"
Jul 21 12:51:02.720578 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:51:02.720550    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h5jbq\" (UniqueName: \"kubernetes.io/projected/067dd5c2-04fa-471c-bdd3-652ea5c57333-kube-api-access-h5jbq\") pod \"e2e-distinct-simulated-kserve-59975d69f9-ng878\" (UID: \"067dd5c2-04fa-471c-bdd3-652ea5c57333\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ng878"
Jul 21 12:51:02.720578 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:51:02.720568    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/067dd5c2-04fa-471c-bdd3-652ea5c57333-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-ng878\" (UID: \"067dd5c2-04fa-471c-bdd3-652ea5c57333\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ng878"
Jul 21 12:51:02.720820 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:51:02.720676    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/067dd5c2-04fa-471c-bdd3-652ea5c57333-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-ng878\" (UID: \"067dd5c2-04fa-471c-bdd3-652ea5c57333\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ng878"
Jul 21 12:51:02.720820 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:51:02.720701    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/067dd5c2-04fa-471c-bdd3-652ea5c57333-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-ng878\" (UID: \"067dd5c2-04fa-471c-bdd3-652ea5c57333\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ng878"
Jul 21 12:51:02.720820 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:51:02.720722    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/067dd5c2-04fa-471c-bdd3-652ea5c57333-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-ng878\" (UID: \"067dd5c2-04fa-471c-bdd3-652ea5c57333\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ng878"
Jul 21 12:51:02.720972 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:51:02.720891    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/067dd5c2-04fa-471c-bdd3-652ea5c57333-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-ng878\" (UID: \"067dd5c2-04fa-471c-bdd3-652ea5c57333\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ng878"
Jul 21 12:51:02.721069 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:51:02.720977    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/067dd5c2-04fa-471c-bdd3-652ea5c57333-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-ng878\" (UID: \"067dd5c2-04fa-471c-bdd3-652ea5c57333\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ng878"
Jul 21 12:51:02.721069 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:51:02.721028    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/067dd5c2-04fa-471c-bdd3-652ea5c57333-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-ng878\" (UID: \"067dd5c2-04fa-471c-bdd3-652ea5c57333\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ng878"
Jul 21 12:51:02.722985 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:51:02.722964    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/067dd5c2-04fa-471c-bdd3-652ea5c57333-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-ng878\" (UID: \"067dd5c2-04fa-471c-bdd3-652ea5c57333\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ng878"
Jul 21 12:51:02.723381 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:51:02.723362    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/067dd5c2-04fa-471c-bdd3-652ea5c57333-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-ng878\" (UID: \"067dd5c2-04fa-471c-bdd3-652ea5c57333\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ng878"
Jul 21 12:51:02.729356 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:51:02.729334    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h5jbq\" (UniqueName: \"kubernetes.io/projected/067dd5c2-04fa-471c-bdd3-652ea5c57333-kube-api-access-h5jbq\") pod \"e2e-distinct-simulated-kserve-59975d69f9-ng878\" (UID: \"067dd5c2-04fa-471c-bdd3-652ea5c57333\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ng878"
Jul 21 12:51:02.776164 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:51:02.776139    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ng878"
Jul 21 12:51:02.914715 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:51:02.914670    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-ng878"]
Jul 21 12:51:02.916105 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:51:02.916076    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod067dd5c2_04fa_471c_bdd3_652ea5c57333.slice/crio-640be5855a9a3f21a16530ca3dbbbf07d728a085f4bd8b790e6da2017166fa93 WatchSource:0}: Error finding container 640be5855a9a3f21a16530ca3dbbbf07d728a085f4bd8b790e6da2017166fa93: Status 404 returned error can't find the container with id 640be5855a9a3f21a16530ca3dbbbf07d728a085f4bd8b790e6da2017166fa93
Jul 21 12:51:02.917543 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:51:02.917525    2556 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 21 12:51:03.481168 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:51:03.481130    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ng878" event={"ID":"067dd5c2-04fa-471c-bdd3-652ea5c57333","Type":"ContainerStarted","Data":"640be5855a9a3f21a16530ca3dbbbf07d728a085f4bd8b790e6da2017166fa93"}
Jul 21 12:51:05.493761 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:51:05.493729    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ng878" event={"ID":"067dd5c2-04fa-471c-bdd3-652ea5c57333","Type":"ContainerStarted","Data":"680819bcdd5f03b2fc0b74609cab5db0e01b1bd81211b4aea7d14ef7e8e90952"}
Jul 21 12:51:05.516822 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:51:05.516774    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ng878" podStartSLOduration=1.379469992 podStartE2EDuration="3.516760016s" totalImagesPullingTime="2.137290024s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:51:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:51:05.515358988 +0000 UTC m=+678.819364098" watchObservedRunningTime="2026-07-21 12:51:05.516760016 +0000 UTC m=+678.820765106"
Jul 21 12:51:12.777069 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:51:12.777007    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ng878"
Jul 21 12:51:12.777561 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:51:12.777148    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ng878"
Jul 21 12:51:12.789427 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:51:12.789399    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ng878"
Jul 21 12:51:13.547602 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:51:13.547571    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-ng878"
Jul 21 12:51:39.064285 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:51:39.064212    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-9skqr"]
Jul 21 12:51:39.070485 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:51:39.070466    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9skqr"
Jul 21 12:51:39.073723 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:51:39.073705    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Jul 21 12:51:39.079123 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:51:39.079103    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-9skqr"]
Jul 21 12:51:39.125410 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:51:39.125380    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bb3d9bf3-7469-41d0-bc43-1e251e23618d-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9skqr\" (UID: \"bb3d9bf3-7469-41d0-bc43-1e251e23618d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9skqr"
Jul 21 12:51:39.125549 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:51:39.125441    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bb3d9bf3-7469-41d0-bc43-1e251e23618d-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9skqr\" (UID: \"bb3d9bf3-7469-41d0-bc43-1e251e23618d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9skqr"
Jul 21 12:51:39.125549 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:51:39.125459    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bb3d9bf3-7469-41d0-bc43-1e251e23618d-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9skqr\" (UID: \"bb3d9bf3-7469-41d0-bc43-1e251e23618d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9skqr"
Jul 21 12:51:39.125549 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:51:39.125504    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bb3d9bf3-7469-41d0-bc43-1e251e23618d-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9skqr\" (UID: \"bb3d9bf3-7469-41d0-bc43-1e251e23618d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9skqr"
Jul 21 12:51:39.125549 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:51:39.125546    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c48nr\" (UniqueName: \"kubernetes.io/projected/bb3d9bf3-7469-41d0-bc43-1e251e23618d-kube-api-access-c48nr\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9skqr\" (UID: \"bb3d9bf3-7469-41d0-bc43-1e251e23618d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9skqr"
Jul 21 12:51:39.125787 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:51:39.125593    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bb3d9bf3-7469-41d0-bc43-1e251e23618d-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9skqr\" (UID: \"bb3d9bf3-7469-41d0-bc43-1e251e23618d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9skqr"
Jul 21 12:51:39.226153 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:51:39.226129    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bb3d9bf3-7469-41d0-bc43-1e251e23618d-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9skqr\" (UID: \"bb3d9bf3-7469-41d0-bc43-1e251e23618d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9skqr"
Jul 21 12:51:39.226307 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:51:39.226159    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bb3d9bf3-7469-41d0-bc43-1e251e23618d-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9skqr\" (UID: \"bb3d9bf3-7469-41d0-bc43-1e251e23618d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9skqr"
Jul 21 12:51:39.226307 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:51:39.226178    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bb3d9bf3-7469-41d0-bc43-1e251e23618d-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9skqr\" (UID: \"bb3d9bf3-7469-41d0-bc43-1e251e23618d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9skqr"
Jul 21 12:51:39.226307 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:51:39.226211    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c48nr\" (UniqueName: \"kubernetes.io/projected/bb3d9bf3-7469-41d0-bc43-1e251e23618d-kube-api-access-c48nr\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9skqr\" (UID: \"bb3d9bf3-7469-41d0-bc43-1e251e23618d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9skqr"
Jul 21 12:51:39.226307 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:51:39.226252    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bb3d9bf3-7469-41d0-bc43-1e251e23618d-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9skqr\" (UID: \"bb3d9bf3-7469-41d0-bc43-1e251e23618d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9skqr"
Jul 21 12:51:39.226307 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:51:39.226305    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bb3d9bf3-7469-41d0-bc43-1e251e23618d-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9skqr\" (UID: \"bb3d9bf3-7469-41d0-bc43-1e251e23618d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9skqr"
Jul 21 12:51:39.226578 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:51:39.226546    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bb3d9bf3-7469-41d0-bc43-1e251e23618d-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9skqr\" (UID: \"bb3d9bf3-7469-41d0-bc43-1e251e23618d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9skqr"
Jul 21 12:51:39.226578 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:51:39.226564    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bb3d9bf3-7469-41d0-bc43-1e251e23618d-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9skqr\" (UID: \"bb3d9bf3-7469-41d0-bc43-1e251e23618d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9skqr"
Jul 21 12:51:39.226669 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:51:39.226620    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bb3d9bf3-7469-41d0-bc43-1e251e23618d-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9skqr\" (UID: \"bb3d9bf3-7469-41d0-bc43-1e251e23618d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9skqr"
Jul 21 12:51:39.228532 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:51:39.228512    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bb3d9bf3-7469-41d0-bc43-1e251e23618d-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9skqr\" (UID: \"bb3d9bf3-7469-41d0-bc43-1e251e23618d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9skqr"
Jul 21 12:51:39.228776 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:51:39.228757    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bb3d9bf3-7469-41d0-bc43-1e251e23618d-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9skqr\" (UID: \"bb3d9bf3-7469-41d0-bc43-1e251e23618d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9skqr"
Jul 21 12:51:39.241672 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:51:39.241652    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c48nr\" (UniqueName: \"kubernetes.io/projected/bb3d9bf3-7469-41d0-bc43-1e251e23618d-kube-api-access-c48nr\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9skqr\" (UID: \"bb3d9bf3-7469-41d0-bc43-1e251e23618d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9skqr"
Jul 21 12:51:39.381750 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:51:39.381660    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9skqr"
Jul 21 12:51:39.522421 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:51:39.522394    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-9skqr"]
Jul 21 12:51:39.524136 ip-10-0-130-2 kubenswrapper[2556]: W0721 12:51:39.524112    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb3d9bf3_7469_41d0_bc43_1e251e23618d.slice/crio-3bf439b88bdd565ee101da08485fb97d44f2483eabffbe11cb00a28e9feb4f5f WatchSource:0}: Error finding container 3bf439b88bdd565ee101da08485fb97d44f2483eabffbe11cb00a28e9feb4f5f: Status 404 returned error can't find the container with id 3bf439b88bdd565ee101da08485fb97d44f2483eabffbe11cb00a28e9feb4f5f
Jul 21 12:51:39.670086 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:51:39.669988    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9skqr" event={"ID":"bb3d9bf3-7469-41d0-bc43-1e251e23618d","Type":"ContainerStarted","Data":"3bf439b88bdd565ee101da08485fb97d44f2483eabffbe11cb00a28e9feb4f5f"}
Jul 21 12:51:40.677196 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:51:40.677163    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9skqr" event={"ID":"bb3d9bf3-7469-41d0-bc43-1e251e23618d","Type":"ContainerStarted","Data":"e3d3c9272769123b5e76b4e0c920a0e8fa1292178822a97674e959a750060b10"}
Jul 21 12:51:40.699981 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:51:40.699937    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9skqr" podStartSLOduration=1.259463799 podStartE2EDuration="1.699922033s" totalImagesPullingTime="440.458234ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:51:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:51:40.697541945 +0000 UTC m=+714.001547062" watchObservedRunningTime="2026-07-21 12:51:40.699922033 +0000 UTC m=+714.003927122"
Jul 21 12:51:49.381861 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:51:49.381813    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9skqr"
Jul 21 12:51:49.382390 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:51:49.381966    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9skqr"
Jul 21 12:51:49.394500 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:51:49.394469    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9skqr"
Jul 21 12:51:49.741548 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:51:49.741522    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9skqr"
Jul 21 12:53:01.361307 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:53:01.361266    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7bb9f58598-mssv5"]
Jul 21 12:53:01.361912 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:53:01.361726    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-7bb9f58598-mssv5" podUID="2b659e93-adc5-433a-9074-fd4a5b285e74" containerName="authorino" containerID="cri-o://0a1c67e1c9153154dac17c12146992a5b369805156ee430d2561dbfdbcc71f1e" gracePeriod=30
Jul 21 12:53:01.608647 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:53:01.608624    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bb9f58598-mssv5"
Jul 21 12:53:01.654866 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:53:01.654792    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/2b659e93-adc5-433a-9074-fd4a5b285e74-tls-cert\") pod \"2b659e93-adc5-433a-9074-fd4a5b285e74\" (UID: \"2b659e93-adc5-433a-9074-fd4a5b285e74\") "
Jul 21 12:53:01.655005 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:53:01.654987    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s6nxt\" (UniqueName: \"kubernetes.io/projected/2b659e93-adc5-433a-9074-fd4a5b285e74-kube-api-access-s6nxt\") pod \"2b659e93-adc5-433a-9074-fd4a5b285e74\" (UID: \"2b659e93-adc5-433a-9074-fd4a5b285e74\") "
Jul 21 12:53:01.657048 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:53:01.656993    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2b659e93-adc5-433a-9074-fd4a5b285e74-kube-api-access-s6nxt" (OuterVolumeSpecName: "kube-api-access-s6nxt") pod "2b659e93-adc5-433a-9074-fd4a5b285e74" (UID: "2b659e93-adc5-433a-9074-fd4a5b285e74"). InnerVolumeSpecName "kube-api-access-s6nxt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 12:53:01.667918 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:53:01.667895    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2b659e93-adc5-433a-9074-fd4a5b285e74-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "2b659e93-adc5-433a-9074-fd4a5b285e74" (UID: "2b659e93-adc5-433a-9074-fd4a5b285e74"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 12:53:01.756650 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:53:01.756627    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s6nxt\" (UniqueName: \"kubernetes.io/projected/2b659e93-adc5-433a-9074-fd4a5b285e74-kube-api-access-s6nxt\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Jul 21 12:53:01.756650 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:53:01.756651    2556 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/2b659e93-adc5-433a-9074-fd4a5b285e74-tls-cert\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Jul 21 12:53:02.109506 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:53:02.109471    2556 generic.go:356] "Generic (PLEG): container finished" podID="2b659e93-adc5-433a-9074-fd4a5b285e74" containerID="0a1c67e1c9153154dac17c12146992a5b369805156ee430d2561dbfdbcc71f1e" exitCode=0
Jul 21 12:53:02.109670 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:53:02.109521    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bb9f58598-mssv5"
Jul 21 12:53:02.109670 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:53:02.109529    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bb9f58598-mssv5" event={"ID":"2b659e93-adc5-433a-9074-fd4a5b285e74","Type":"ContainerDied","Data":"0a1c67e1c9153154dac17c12146992a5b369805156ee430d2561dbfdbcc71f1e"}
Jul 21 12:53:02.109670 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:53:02.109592    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bb9f58598-mssv5" event={"ID":"2b659e93-adc5-433a-9074-fd4a5b285e74","Type":"ContainerDied","Data":"e126d99239819ea3557d2ac00e4fe72fa744b3b5817f59378afd11a8cb464cb0"}
Jul 21 12:53:02.109670 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:53:02.109620    2556 scope.go:117] "RemoveContainer" containerID="0a1c67e1c9153154dac17c12146992a5b369805156ee430d2561dbfdbcc71f1e"
Jul 21 12:53:02.119954 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:53:02.119937    2556 scope.go:117] "RemoveContainer" containerID="0a1c67e1c9153154dac17c12146992a5b369805156ee430d2561dbfdbcc71f1e"
Jul 21 12:53:02.120244 ip-10-0-130-2 kubenswrapper[2556]: E0721 12:53:02.120222    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0a1c67e1c9153154dac17c12146992a5b369805156ee430d2561dbfdbcc71f1e\": container with ID starting with 0a1c67e1c9153154dac17c12146992a5b369805156ee430d2561dbfdbcc71f1e not found: ID does not exist" containerID="0a1c67e1c9153154dac17c12146992a5b369805156ee430d2561dbfdbcc71f1e"
Jul 21 12:53:02.120328 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:53:02.120257    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0a1c67e1c9153154dac17c12146992a5b369805156ee430d2561dbfdbcc71f1e"} err="failed to get container status \"0a1c67e1c9153154dac17c12146992a5b369805156ee430d2561dbfdbcc71f1e\": rpc error: code = NotFound desc = could not find container \"0a1c67e1c9153154dac17c12146992a5b369805156ee430d2561dbfdbcc71f1e\": container with ID starting with 0a1c67e1c9153154dac17c12146992a5b369805156ee430d2561dbfdbcc71f1e not found: ID does not exist"
Jul 21 12:53:02.135136 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:53:02.135107    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7bb9f58598-mssv5"]
Jul 21 12:53:02.138121 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:53:02.138100    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7bb9f58598-mssv5"]
Jul 21 12:53:03.194629 ip-10-0-130-2 kubenswrapper[2556]: I0721 12:53:03.194594    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2b659e93-adc5-433a-9074-fd4a5b285e74" path="/var/lib/kubelet/pods/2b659e93-adc5-433a-9074-fd4a5b285e74/volumes"
Jul 21 13:00:00.157963 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:00:00.157932    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29743980-dk8kb"]
Jul 21 13:00:00.158422 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:00:00.158378    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2b659e93-adc5-433a-9074-fd4a5b285e74" containerName="authorino"
Jul 21 13:00:00.158422 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:00:00.158390    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="2b659e93-adc5-433a-9074-fd4a5b285e74" containerName="authorino"
Jul 21 13:00:00.158499 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:00:00.158468    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="2b659e93-adc5-433a-9074-fd4a5b285e74" containerName="authorino"
Jul 21 13:00:00.161502 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:00:00.161487    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29743980-dk8kb"
Jul 21 13:00:00.163959 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:00:00.163934    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5nsrr\""
Jul 21 13:00:00.170875 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:00:00.170857    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29743980-dk8kb"
Jul 21 13:00:00.180076 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:00:00.180047    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29743980-dk8kb"]
Jul 21 13:00:00.514550 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:00:00.514528    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29743980-dk8kb"]
Jul 21 13:00:00.516632 ip-10-0-130-2 kubenswrapper[2556]: W0721 13:00:00.516605    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99fdf9c5_69d0_443c_ae68_04003cd37a46.slice/crio-4cda3b46f722253a94b50033cbf03cfcc98037ae301abcb77146015a0be78903 WatchSource:0}: Error finding container 4cda3b46f722253a94b50033cbf03cfcc98037ae301abcb77146015a0be78903: Status 404 returned error can't find the container with id 4cda3b46f722253a94b50033cbf03cfcc98037ae301abcb77146015a0be78903
Jul 21 13:00:00.518508 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:00:00.518489    2556 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 21 13:00:01.306346 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:00:01.306313    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29743980-dk8kb" event={"ID":"99fdf9c5-69d0-443c-ae68-04003cd37a46","Type":"ContainerStarted","Data":"4cda3b46f722253a94b50033cbf03cfcc98037ae301abcb77146015a0be78903"}
Jul 21 13:00:07.339699 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:00:07.339669    2556 generic.go:356] "Generic (PLEG): container finished" podID="99fdf9c5-69d0-443c-ae68-04003cd37a46" containerID="c9d0b0e80eb89f10071cc269d0519d247607302ffbfde86889c9fb6c267ebaf1" exitCode=0
Jul 21 13:00:07.340035 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:00:07.339723    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29743980-dk8kb" event={"ID":"99fdf9c5-69d0-443c-ae68-04003cd37a46","Type":"ContainerDied","Data":"c9d0b0e80eb89f10071cc269d0519d247607302ffbfde86889c9fb6c267ebaf1"}
Jul 21 13:00:08.488476 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:00:08.488452    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29743980-dk8kb"
Jul 21 13:00:09.351520 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:00:09.351493    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29743980-dk8kb"
Jul 21 13:00:09.351520 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:00:09.351502    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29743980-dk8kb" event={"ID":"99fdf9c5-69d0-443c-ae68-04003cd37a46","Type":"ContainerDied","Data":"4cda3b46f722253a94b50033cbf03cfcc98037ae301abcb77146015a0be78903"}
Jul 21 13:00:09.351705 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:00:09.351548    2556 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4cda3b46f722253a94b50033cbf03cfcc98037ae301abcb77146015a0be78903"
Jul 21 13:08:10.848676 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:08:10.848592    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-e773b8a4-openshift-default-6455b8c6c7-95ssv"]
Jul 21 13:08:10.851193 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:08:10.849079    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="99fdf9c5-69d0-443c-ae68-04003cd37a46" containerName="cleanup"
Jul 21 13:08:10.851193 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:08:10.849097    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="99fdf9c5-69d0-443c-ae68-04003cd37a46" containerName="cleanup"
Jul 21 13:08:10.851193 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:08:10.849193    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="99fdf9c5-69d0-443c-ae68-04003cd37a46" containerName="cleanup"
Jul 21 13:08:10.852179 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:08:10.852164    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-e773b8a4-openshift-default-6455b8c6c7-95ssv"
Jul 21 13:08:10.854252 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:08:10.854233    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-e773b8a4-openshift-default-dockercfg-kq8r6\""
Jul 21 13:08:10.871203 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:08:10.871175    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-e773b8a4-openshift-default-6455b8c6c7-95ssv"]
Jul 21 13:08:10.981936 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:08:10.981896    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3ce5ffbf-b61f-4b6e-bcd9-4626d55daaa2-istio-envoy\") pod \"e2e-ait-e773b8a4-openshift-default-6455b8c6c7-95ssv\" (UID: \"3ce5ffbf-b61f-4b6e-bcd9-4626d55daaa2\") " pod="openshift-ingress/e2e-ait-e773b8a4-openshift-default-6455b8c6c7-95ssv"
Jul 21 13:08:10.982151 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:08:10.981965    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3ce5ffbf-b61f-4b6e-bcd9-4626d55daaa2-workload-certs\") pod \"e2e-ait-e773b8a4-openshift-default-6455b8c6c7-95ssv\" (UID: \"3ce5ffbf-b61f-4b6e-bcd9-4626d55daaa2\") " pod="openshift-ingress/e2e-ait-e773b8a4-openshift-default-6455b8c6c7-95ssv"
Jul 21 13:08:10.982151 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:08:10.982073    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3ce5ffbf-b61f-4b6e-bcd9-4626d55daaa2-istiod-ca-cert\") pod \"e2e-ait-e773b8a4-openshift-default-6455b8c6c7-95ssv\" (UID: \"3ce5ffbf-b61f-4b6e-bcd9-4626d55daaa2\") " pod="openshift-ingress/e2e-ait-e773b8a4-openshift-default-6455b8c6c7-95ssv"
Jul 21 13:08:10.982151 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:08:10.982109    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ce5ffbf-b61f-4b6e-bcd9-4626d55daaa2-credential-socket\") pod \"e2e-ait-e773b8a4-openshift-default-6455b8c6c7-95ssv\" (UID: \"3ce5ffbf-b61f-4b6e-bcd9-4626d55daaa2\") " pod="openshift-ingress/e2e-ait-e773b8a4-openshift-default-6455b8c6c7-95ssv"
Jul 21 13:08:10.982313 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:08:10.982172    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3ce5ffbf-b61f-4b6e-bcd9-4626d55daaa2-istio-token\") pod \"e2e-ait-e773b8a4-openshift-default-6455b8c6c7-95ssv\" (UID: \"3ce5ffbf-b61f-4b6e-bcd9-4626d55daaa2\") " pod="openshift-ingress/e2e-ait-e773b8a4-openshift-default-6455b8c6c7-95ssv"
Jul 21 13:08:10.982313 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:08:10.982210    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ce5ffbf-b61f-4b6e-bcd9-4626d55daaa2-workload-socket\") pod \"e2e-ait-e773b8a4-openshift-default-6455b8c6c7-95ssv\" (UID: \"3ce5ffbf-b61f-4b6e-bcd9-4626d55daaa2\") " pod="openshift-ingress/e2e-ait-e773b8a4-openshift-default-6455b8c6c7-95ssv"
Jul 21 13:08:10.982313 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:08:10.982231    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bsjqh\" (UniqueName: \"kubernetes.io/projected/3ce5ffbf-b61f-4b6e-bcd9-4626d55daaa2-kube-api-access-bsjqh\") pod \"e2e-ait-e773b8a4-openshift-default-6455b8c6c7-95ssv\" (UID: \"3ce5ffbf-b61f-4b6e-bcd9-4626d55daaa2\") " pod="openshift-ingress/e2e-ait-e773b8a4-openshift-default-6455b8c6c7-95ssv"
Jul 21 13:08:10.982313 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:08:10.982297    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3ce5ffbf-b61f-4b6e-bcd9-4626d55daaa2-istio-data\") pod \"e2e-ait-e773b8a4-openshift-default-6455b8c6c7-95ssv\" (UID: \"3ce5ffbf-b61f-4b6e-bcd9-4626d55daaa2\") " pod="openshift-ingress/e2e-ait-e773b8a4-openshift-default-6455b8c6c7-95ssv"
Jul 21 13:08:10.982313 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:08:10.982313    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3ce5ffbf-b61f-4b6e-bcd9-4626d55daaa2-istio-podinfo\") pod \"e2e-ait-e773b8a4-openshift-default-6455b8c6c7-95ssv\" (UID: \"3ce5ffbf-b61f-4b6e-bcd9-4626d55daaa2\") " pod="openshift-ingress/e2e-ait-e773b8a4-openshift-default-6455b8c6c7-95ssv"
Jul 21 13:08:11.083683 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:08:11.083657    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3ce5ffbf-b61f-4b6e-bcd9-4626d55daaa2-istio-data\") pod \"e2e-ait-e773b8a4-openshift-default-6455b8c6c7-95ssv\" (UID: \"3ce5ffbf-b61f-4b6e-bcd9-4626d55daaa2\") " pod="openshift-ingress/e2e-ait-e773b8a4-openshift-default-6455b8c6c7-95ssv"
Jul 21 13:08:11.083811 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:08:11.083689    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3ce5ffbf-b61f-4b6e-bcd9-4626d55daaa2-istio-podinfo\") pod \"e2e-ait-e773b8a4-openshift-default-6455b8c6c7-95ssv\" (UID: \"3ce5ffbf-b61f-4b6e-bcd9-4626d55daaa2\") " pod="openshift-ingress/e2e-ait-e773b8a4-openshift-default-6455b8c6c7-95ssv"
Jul 21 13:08:11.083811 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:08:11.083716    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3ce5ffbf-b61f-4b6e-bcd9-4626d55daaa2-istio-envoy\") pod \"e2e-ait-e773b8a4-openshift-default-6455b8c6c7-95ssv\" (UID: \"3ce5ffbf-b61f-4b6e-bcd9-4626d55daaa2\") " pod="openshift-ingress/e2e-ait-e773b8a4-openshift-default-6455b8c6c7-95ssv"
Jul 21 13:08:11.083811 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:08:11.083735    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3ce5ffbf-b61f-4b6e-bcd9-4626d55daaa2-workload-certs\") pod \"e2e-ait-e773b8a4-openshift-default-6455b8c6c7-95ssv\" (UID: \"3ce5ffbf-b61f-4b6e-bcd9-4626d55daaa2\") " pod="openshift-ingress/e2e-ait-e773b8a4-openshift-default-6455b8c6c7-95ssv"
Jul 21 13:08:11.083811 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:08:11.083802    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3ce5ffbf-b61f-4b6e-bcd9-4626d55daaa2-istiod-ca-cert\") pod \"e2e-ait-e773b8a4-openshift-default-6455b8c6c7-95ssv\" (UID: \"3ce5ffbf-b61f-4b6e-bcd9-4626d55daaa2\") " pod="openshift-ingress/e2e-ait-e773b8a4-openshift-default-6455b8c6c7-95ssv"
Jul 21 13:08:11.084036 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:08:11.083835    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ce5ffbf-b61f-4b6e-bcd9-4626d55daaa2-credential-socket\") pod \"e2e-ait-e773b8a4-openshift-default-6455b8c6c7-95ssv\" (UID: \"3ce5ffbf-b61f-4b6e-bcd9-4626d55daaa2\") " pod="openshift-ingress/e2e-ait-e773b8a4-openshift-default-6455b8c6c7-95ssv"
Jul 21 13:08:11.084036 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:08:11.083873    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3ce5ffbf-b61f-4b6e-bcd9-4626d55daaa2-istio-token\") pod \"e2e-ait-e773b8a4-openshift-default-6455b8c6c7-95ssv\" (UID: \"3ce5ffbf-b61f-4b6e-bcd9-4626d55daaa2\") " pod="openshift-ingress/e2e-ait-e773b8a4-openshift-default-6455b8c6c7-95ssv"
Jul 21 13:08:11.084036 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:08:11.083898    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ce5ffbf-b61f-4b6e-bcd9-4626d55daaa2-workload-socket\") pod \"e2e-ait-e773b8a4-openshift-default-6455b8c6c7-95ssv\" (UID: \"3ce5ffbf-b61f-4b6e-bcd9-4626d55daaa2\") " pod="openshift-ingress/e2e-ait-e773b8a4-openshift-default-6455b8c6c7-95ssv"
Jul 21 13:08:11.084036 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:08:11.083920    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bsjqh\" (UniqueName: \"kubernetes.io/projected/3ce5ffbf-b61f-4b6e-bcd9-4626d55daaa2-kube-api-access-bsjqh\") pod \"e2e-ait-e773b8a4-openshift-default-6455b8c6c7-95ssv\" (UID: \"3ce5ffbf-b61f-4b6e-bcd9-4626d55daaa2\") " pod="openshift-ingress/e2e-ait-e773b8a4-openshift-default-6455b8c6c7-95ssv"
Jul 21 13:08:11.084260 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:08:11.084118    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3ce5ffbf-b61f-4b6e-bcd9-4626d55daaa2-istio-data\") pod \"e2e-ait-e773b8a4-openshift-default-6455b8c6c7-95ssv\" (UID: \"3ce5ffbf-b61f-4b6e-bcd9-4626d55daaa2\") " pod="openshift-ingress/e2e-ait-e773b8a4-openshift-default-6455b8c6c7-95ssv"
Jul 21 13:08:11.084260 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:08:11.084178    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3ce5ffbf-b61f-4b6e-bcd9-4626d55daaa2-workload-certs\") pod \"e2e-ait-e773b8a4-openshift-default-6455b8c6c7-95ssv\" (UID: \"3ce5ffbf-b61f-4b6e-bcd9-4626d55daaa2\") " pod="openshift-ingress/e2e-ait-e773b8a4-openshift-default-6455b8c6c7-95ssv"
Jul 21 13:08:11.084368 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:08:11.084259    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ce5ffbf-b61f-4b6e-bcd9-4626d55daaa2-credential-socket\") pod \"e2e-ait-e773b8a4-openshift-default-6455b8c6c7-95ssv\" (UID: \"3ce5ffbf-b61f-4b6e-bcd9-4626d55daaa2\") " pod="openshift-ingress/e2e-ait-e773b8a4-openshift-default-6455b8c6c7-95ssv"
Jul 21 13:08:11.084459 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:08:11.084438    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ce5ffbf-b61f-4b6e-bcd9-4626d55daaa2-workload-socket\") pod \"e2e-ait-e773b8a4-openshift-default-6455b8c6c7-95ssv\" (UID: \"3ce5ffbf-b61f-4b6e-bcd9-4626d55daaa2\") " pod="openshift-ingress/e2e-ait-e773b8a4-openshift-default-6455b8c6c7-95ssv"
Jul 21 13:08:11.084804 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:08:11.084783    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3ce5ffbf-b61f-4b6e-bcd9-4626d55daaa2-istiod-ca-cert\") pod \"e2e-ait-e773b8a4-openshift-default-6455b8c6c7-95ssv\" (UID: \"3ce5ffbf-b61f-4b6e-bcd9-4626d55daaa2\") " pod="openshift-ingress/e2e-ait-e773b8a4-openshift-default-6455b8c6c7-95ssv"
Jul 21 13:08:11.085888 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:08:11.085870    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3ce5ffbf-b61f-4b6e-bcd9-4626d55daaa2-istio-envoy\") pod \"e2e-ait-e773b8a4-openshift-default-6455b8c6c7-95ssv\" (UID: \"3ce5ffbf-b61f-4b6e-bcd9-4626d55daaa2\") " pod="openshift-ingress/e2e-ait-e773b8a4-openshift-default-6455b8c6c7-95ssv"
Jul 21 13:08:11.086156 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:08:11.086140    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3ce5ffbf-b61f-4b6e-bcd9-4626d55daaa2-istio-podinfo\") pod \"e2e-ait-e773b8a4-openshift-default-6455b8c6c7-95ssv\" (UID: \"3ce5ffbf-b61f-4b6e-bcd9-4626d55daaa2\") " pod="openshift-ingress/e2e-ait-e773b8a4-openshift-default-6455b8c6c7-95ssv"
Jul 21 13:08:11.093143 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:08:11.093116    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3ce5ffbf-b61f-4b6e-bcd9-4626d55daaa2-istio-token\") pod \"e2e-ait-e773b8a4-openshift-default-6455b8c6c7-95ssv\" (UID: \"3ce5ffbf-b61f-4b6e-bcd9-4626d55daaa2\") " pod="openshift-ingress/e2e-ait-e773b8a4-openshift-default-6455b8c6c7-95ssv"
Jul 21 13:08:11.093229 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:08:11.093193    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bsjqh\" (UniqueName: \"kubernetes.io/projected/3ce5ffbf-b61f-4b6e-bcd9-4626d55daaa2-kube-api-access-bsjqh\") pod \"e2e-ait-e773b8a4-openshift-default-6455b8c6c7-95ssv\" (UID: \"3ce5ffbf-b61f-4b6e-bcd9-4626d55daaa2\") " pod="openshift-ingress/e2e-ait-e773b8a4-openshift-default-6455b8c6c7-95ssv"
Jul 21 13:08:11.163136 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:08:11.163081    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-e773b8a4-openshift-default-6455b8c6c7-95ssv"
Jul 21 13:08:11.311311 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:08:11.311286    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-e773b8a4-openshift-default-6455b8c6c7-95ssv"]
Jul 21 13:08:11.313113 ip-10-0-130-2 kubenswrapper[2556]: W0721 13:08:11.313070    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ce5ffbf_b61f_4b6e_bcd9_4626d55daaa2.slice/crio-9476917ec4b795ebce360743c9f2f23fe226ad8ec2319f79bba635e9c029296c WatchSource:0}: Error finding container 9476917ec4b795ebce360743c9f2f23fe226ad8ec2319f79bba635e9c029296c: Status 404 returned error can't find the container with id 9476917ec4b795ebce360743c9f2f23fe226ad8ec2319f79bba635e9c029296c
Jul 21 13:08:11.315121 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:08:11.315091    2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 21 13:08:11.315228 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:08:11.315154    2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 21 13:08:11.315228 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:08:11.315184    2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 21 13:08:11.883805 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:08:11.883770    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e773b8a4-openshift-default-6455b8c6c7-95ssv" event={"ID":"3ce5ffbf-b61f-4b6e-bcd9-4626d55daaa2","Type":"ContainerStarted","Data":"a9d3b724646bb1b7561e4e5f69ad2704f88f792132101e06d57a503c5b6d31b3"}
Jul 21 13:08:11.884243 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:08:11.883819    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e773b8a4-openshift-default-6455b8c6c7-95ssv" event={"ID":"3ce5ffbf-b61f-4b6e-bcd9-4626d55daaa2","Type":"ContainerStarted","Data":"9476917ec4b795ebce360743c9f2f23fe226ad8ec2319f79bba635e9c029296c"}
Jul 21 13:08:11.909070 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:08:11.909004    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-e773b8a4-openshift-default-6455b8c6c7-95ssv" podStartSLOduration=1.90898773 podStartE2EDuration="1.90898773s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 13:08:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:08:11.90713178 +0000 UTC m=+1705.211136888" watchObservedRunningTime="2026-07-21 13:08:11.90898773 +0000 UTC m=+1705.212992819"
Jul 21 13:08:12.163784 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:08:12.163704    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-e773b8a4-openshift-default-6455b8c6c7-95ssv"
Jul 21 13:08:12.168746 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:08:12.168722    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-e773b8a4-openshift-default-6455b8c6c7-95ssv"
Jul 21 13:08:12.889264 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:08:12.889236    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-e773b8a4-openshift-default-6455b8c6c7-95ssv"
Jul 21 13:08:12.890285 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:08:12.890266    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-e773b8a4-openshift-default-6455b8c6c7-95ssv"
Jul 21 13:08:27.891126 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:08:27.891094    2556 patch_prober.go:28] interesting pod/e2e-ait-e773b8a4-openshift-default-6455b8c6c7-95ssv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Jul 21 13:08:27.891546 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:08:27.891148    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-e773b8a4-openshift-default-6455b8c6c7-95ssv" podUID="3ce5ffbf-b61f-4b6e-bcd9-4626d55daaa2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Jul 21 13:08:27.979329 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:08:27.979300    2556 generic.go:356] "Generic (PLEG): container finished" podID="3ce5ffbf-b61f-4b6e-bcd9-4626d55daaa2" containerID="a9d3b724646bb1b7561e4e5f69ad2704f88f792132101e06d57a503c5b6d31b3" exitCode=0
Jul 21 13:08:27.979462 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:08:27.979376    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e773b8a4-openshift-default-6455b8c6c7-95ssv" event={"ID":"3ce5ffbf-b61f-4b6e-bcd9-4626d55daaa2","Type":"ContainerDied","Data":"a9d3b724646bb1b7561e4e5f69ad2704f88f792132101e06d57a503c5b6d31b3"}
Jul 21 13:08:27.979758 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:08:27.979744    2556 scope.go:117] "RemoveContainer" containerID="a9d3b724646bb1b7561e4e5f69ad2704f88f792132101e06d57a503c5b6d31b3"
Jul 21 13:08:27.980743 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:08:27.980713    2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 21 13:08:27.980825 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:08:27.980788    2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 21 13:08:27.980825 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:08:27.980818    2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 21 13:08:28.986839 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:08:28.986798    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e773b8a4-openshift-default-6455b8c6c7-95ssv" event={"ID":"3ce5ffbf-b61f-4b6e-bcd9-4626d55daaa2","Type":"ContainerStarted","Data":"4c3fae2a6f8c3672ca0cabc6441d4adf2cec5093468d93ab65cf16facf80ed3d"}
Jul 21 13:08:29.164074 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:08:29.164038    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-e773b8a4-openshift-default-6455b8c6c7-95ssv"
Jul 21 13:08:29.164280 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:08:29.164257    2556 patch_prober.go:28] interesting pod/e2e-ait-e773b8a4-openshift-default-6455b8c6c7-95ssv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Jul 21 13:08:29.164329 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:08:29.164311    2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-e773b8a4-openshift-default-6455b8c6c7-95ssv" podUID="3ce5ffbf-b61f-4b6e-bcd9-4626d55daaa2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Jul 21 13:08:30.163622 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:08:30.163587    2556 patch_prober.go:28] interesting pod/e2e-ait-e773b8a4-openshift-default-6455b8c6c7-95ssv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Jul 21 13:08:30.163998 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:08:30.163638    2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-e773b8a4-openshift-default-6455b8c6c7-95ssv" podUID="3ce5ffbf-b61f-4b6e-bcd9-4626d55daaa2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Jul 21 13:08:31.164122 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:08:31.164084    2556 patch_prober.go:28] interesting pod/e2e-ait-e773b8a4-openshift-default-6455b8c6c7-95ssv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Jul 21 13:08:31.164510 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:08:31.164135    2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-e773b8a4-openshift-default-6455b8c6c7-95ssv" podUID="3ce5ffbf-b61f-4b6e-bcd9-4626d55daaa2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Jul 21 13:08:32.164403 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:08:32.164358    2556 patch_prober.go:28] interesting pod/e2e-ait-e773b8a4-openshift-default-6455b8c6c7-95ssv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Jul 21 13:08:32.164775 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:08:32.164423    2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-e773b8a4-openshift-default-6455b8c6c7-95ssv" podUID="3ce5ffbf-b61f-4b6e-bcd9-4626d55daaa2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Jul 21 13:08:33.167610 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:08:33.167584    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-e773b8a4-openshift-default-6455b8c6c7-95ssv"
Jul 21 13:08:33.167974 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:08:33.167936    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-e773b8a4-openshift-default-6455b8c6c7-95ssv"
Jul 21 13:08:33.168500 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:08:33.168485    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-e773b8a4-openshift-default-6455b8c6c7-95ssv"
Jul 21 13:08:38.816261 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:08:38.816213    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e773b8a4-qbdct"]
Jul 21 13:08:38.820619 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:08:38.820599    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e773b8a4-qbdct"
Jul 21 13:08:38.822779 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:08:38.822748    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5nsrr\""
Jul 21 13:08:38.828506 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:08:38.828478    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e773b8a4-qbdct"]
Jul 21 13:08:38.940743 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:08:38.940714    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/05db79fe-7e83-45c4-96d3-e68f6dcc000e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-e773b8a4-qbdct\" (UID: \"05db79fe-7e83-45c4-96d3-e68f6dcc000e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e773b8a4-qbdct"
Jul 21 13:08:39.041271 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:08:39.041246    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/05db79fe-7e83-45c4-96d3-e68f6dcc000e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-e773b8a4-qbdct\" (UID: \"05db79fe-7e83-45c4-96d3-e68f6dcc000e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e773b8a4-qbdct"
Jul 21 13:08:39.041875 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:08:39.041841    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/05db79fe-7e83-45c4-96d3-e68f6dcc000e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-e773b8a4-qbdct\" (UID: \"05db79fe-7e83-45c4-96d3-e68f6dcc000e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e773b8a4-qbdct"
Jul 21 13:08:39.134961 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:08:39.134895    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e773b8a4-qbdct"
Jul 21 13:08:39.481814 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:08:39.481786    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e773b8a4-qbdct"]
Jul 21 13:08:39.483382 ip-10-0-130-2 kubenswrapper[2556]: W0721 13:08:39.483358    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05db79fe_7e83_45c4_96d3_e68f6dcc000e.slice/crio-dcf5d9e5b04188beb89416ecd2c417f92284248724cc2a1bff1575637f613551 WatchSource:0}: Error finding container dcf5d9e5b04188beb89416ecd2c417f92284248724cc2a1bff1575637f613551: Status 404 returned error can't find the container with id dcf5d9e5b04188beb89416ecd2c417f92284248724cc2a1bff1575637f613551
Jul 21 13:08:40.056125 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:08:40.056091    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e773b8a4-qbdct" event={"ID":"05db79fe-7e83-45c4-96d3-e68f6dcc000e","Type":"ContainerStarted","Data":"d990b84c107b0c46059d7b9bb2c990abcbdd961a3f50ebfcb45d2dfe9980ca55"}
Jul 21 13:08:40.056457 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:08:40.056147    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e773b8a4-qbdct" event={"ID":"05db79fe-7e83-45c4-96d3-e68f6dcc000e","Type":"ContainerStarted","Data":"dcf5d9e5b04188beb89416ecd2c417f92284248724cc2a1bff1575637f613551"}
Jul 21 13:08:40.074857 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:08:40.074815    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e773b8a4-qbdct" podStartSLOduration=2.074801085 podStartE2EDuration="2.074801085s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 13:08:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:08:40.072380825 +0000 UTC m=+1733.376385915" watchObservedRunningTime="2026-07-21 13:08:40.074801085 +0000 UTC m=+1733.378806175"
Jul 21 13:08:41.063430 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:08:41.063397    2556 generic.go:356] "Generic (PLEG): container finished" podID="05db79fe-7e83-45c4-96d3-e68f6dcc000e" containerID="d990b84c107b0c46059d7b9bb2c990abcbdd961a3f50ebfcb45d2dfe9980ca55" exitCode=0
Jul 21 13:08:41.063830 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:08:41.063464    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e773b8a4-qbdct" event={"ID":"05db79fe-7e83-45c4-96d3-e68f6dcc000e","Type":"ContainerDied","Data":"d990b84c107b0c46059d7b9bb2c990abcbdd961a3f50ebfcb45d2dfe9980ca55"}
Jul 21 13:08:42.206049 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:08:42.206003    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e773b8a4-qbdct"
Jul 21 13:08:42.270648 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:08:42.270624    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/05db79fe-7e83-45c4-96d3-e68f6dcc000e-maas-api-service-ca\") pod \"05db79fe-7e83-45c4-96d3-e68f6dcc000e\" (UID: \"05db79fe-7e83-45c4-96d3-e68f6dcc000e\") "
Jul 21 13:08:42.270947 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:08:42.270927    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/05db79fe-7e83-45c4-96d3-e68f6dcc000e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "05db79fe-7e83-45c4-96d3-e68f6dcc000e" (UID: "05db79fe-7e83-45c4-96d3-e68f6dcc000e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 13:08:42.371680 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:08:42.371620    2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/05db79fe-7e83-45c4-96d3-e68f6dcc000e-maas-api-service-ca\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Jul 21 13:08:43.076620 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:08:43.076592    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e773b8a4-qbdct"
Jul 21 13:08:43.076620 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:08:43.076596    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e773b8a4-qbdct" event={"ID":"05db79fe-7e83-45c4-96d3-e68f6dcc000e","Type":"ContainerDied","Data":"dcf5d9e5b04188beb89416ecd2c417f92284248724cc2a1bff1575637f613551"}
Jul 21 13:08:43.076833 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:08:43.076683    2556 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="dcf5d9e5b04188beb89416ecd2c417f92284248724cc2a1bff1575637f613551"
Jul 21 13:08:53.942623 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:08:53.942589    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e773b8a4-qbdct"]
Jul 21 13:08:53.951486 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:08:53.951448    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e773b8a4-qbdct"]
Jul 21 13:08:54.096189 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:08:54.096158    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-e773b8a4-openshift-default-6455b8c6c7-95ssv"]
Jul 21 13:08:54.096430 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:08:54.096391    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-e773b8a4-openshift-default-6455b8c6c7-95ssv" podUID="3ce5ffbf-b61f-4b6e-bcd9-4626d55daaa2" containerName="istio-proxy" containerID="cri-o://4c3fae2a6f8c3672ca0cabc6441d4adf2cec5093468d93ab65cf16facf80ed3d" gracePeriod=30
Jul 21 13:08:55.196058 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:08:55.196000    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="05db79fe-7e83-45c4-96d3-e68f6dcc000e" path="/var/lib/kubelet/pods/05db79fe-7e83-45c4-96d3-e68f6dcc000e/volumes"
Jul 21 13:08:59.360055 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:08:59.359996    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-e773b8a4-openshift-default-6455b8c6c7-95ssv"
Jul 21 13:08:59.521834 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:08:59.521812    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3ce5ffbf-b61f-4b6e-bcd9-4626d55daaa2-istio-data\") pod \"3ce5ffbf-b61f-4b6e-bcd9-4626d55daaa2\" (UID: \"3ce5ffbf-b61f-4b6e-bcd9-4626d55daaa2\") "
Jul 21 13:08:59.521983 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:08:59.521844    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3ce5ffbf-b61f-4b6e-bcd9-4626d55daaa2-workload-certs\") pod \"3ce5ffbf-b61f-4b6e-bcd9-4626d55daaa2\" (UID: \"3ce5ffbf-b61f-4b6e-bcd9-4626d55daaa2\") "
Jul 21 13:08:59.521983 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:08:59.521895    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ce5ffbf-b61f-4b6e-bcd9-4626d55daaa2-workload-socket\") pod \"3ce5ffbf-b61f-4b6e-bcd9-4626d55daaa2\" (UID: \"3ce5ffbf-b61f-4b6e-bcd9-4626d55daaa2\") "
Jul 21 13:08:59.521983 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:08:59.521942    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3ce5ffbf-b61f-4b6e-bcd9-4626d55daaa2-istio-podinfo\") pod \"3ce5ffbf-b61f-4b6e-bcd9-4626d55daaa2\" (UID: \"3ce5ffbf-b61f-4b6e-bcd9-4626d55daaa2\") "
Jul 21 13:08:59.522203 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:08:59.521996    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ce5ffbf-b61f-4b6e-bcd9-4626d55daaa2-credential-socket\") pod \"3ce5ffbf-b61f-4b6e-bcd9-4626d55daaa2\" (UID: \"3ce5ffbf-b61f-4b6e-bcd9-4626d55daaa2\") "
Jul 21 13:08:59.522203 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:08:59.522054    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bsjqh\" (UniqueName: \"kubernetes.io/projected/3ce5ffbf-b61f-4b6e-bcd9-4626d55daaa2-kube-api-access-bsjqh\") pod \"3ce5ffbf-b61f-4b6e-bcd9-4626d55daaa2\" (UID: \"3ce5ffbf-b61f-4b6e-bcd9-4626d55daaa2\") "
Jul 21 13:08:59.522203 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:08:59.522085    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3ce5ffbf-b61f-4b6e-bcd9-4626d55daaa2-istio-envoy\") pod \"3ce5ffbf-b61f-4b6e-bcd9-4626d55daaa2\" (UID: \"3ce5ffbf-b61f-4b6e-bcd9-4626d55daaa2\") "
Jul 21 13:08:59.522203 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:08:59.522175    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ce5ffbf-b61f-4b6e-bcd9-4626d55daaa2-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "3ce5ffbf-b61f-4b6e-bcd9-4626d55daaa2" (UID: "3ce5ffbf-b61f-4b6e-bcd9-4626d55daaa2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 13:08:59.522203 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:08:59.522195    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3ce5ffbf-b61f-4b6e-bcd9-4626d55daaa2-istiod-ca-cert\") pod \"3ce5ffbf-b61f-4b6e-bcd9-4626d55daaa2\" (UID: \"3ce5ffbf-b61f-4b6e-bcd9-4626d55daaa2\") "
Jul 21 13:08:59.522424 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:08:59.522225    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3ce5ffbf-b61f-4b6e-bcd9-4626d55daaa2-istio-token\") pod \"3ce5ffbf-b61f-4b6e-bcd9-4626d55daaa2\" (UID: \"3ce5ffbf-b61f-4b6e-bcd9-4626d55daaa2\") "
Jul 21 13:08:59.522424 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:08:59.522249    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ce5ffbf-b61f-4b6e-bcd9-4626d55daaa2-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "3ce5ffbf-b61f-4b6e-bcd9-4626d55daaa2" (UID: "3ce5ffbf-b61f-4b6e-bcd9-4626d55daaa2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 13:08:59.522519 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:08:59.522472    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3ce5ffbf-b61f-4b6e-bcd9-4626d55daaa2-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "3ce5ffbf-b61f-4b6e-bcd9-4626d55daaa2" (UID: "3ce5ffbf-b61f-4b6e-bcd9-4626d55daaa2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 13:08:59.522583 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:08:59.522526    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ce5ffbf-b61f-4b6e-bcd9-4626d55daaa2-istio-data" (OuterVolumeSpecName: "istio-data") pod "3ce5ffbf-b61f-4b6e-bcd9-4626d55daaa2" (UID: "3ce5ffbf-b61f-4b6e-bcd9-4626d55daaa2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 13:08:59.522638 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:08:59.522586    2556 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3ce5ffbf-b61f-4b6e-bcd9-4626d55daaa2-istiod-ca-cert\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Jul 21 13:08:59.522714 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:08:59.522688    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ce5ffbf-b61f-4b6e-bcd9-4626d55daaa2-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "3ce5ffbf-b61f-4b6e-bcd9-4626d55daaa2" (UID: "3ce5ffbf-b61f-4b6e-bcd9-4626d55daaa2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 13:08:59.522889 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:08:59.522699    2556 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3ce5ffbf-b61f-4b6e-bcd9-4626d55daaa2-workload-certs\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Jul 21 13:08:59.522889 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:08:59.522745    2556 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ce5ffbf-b61f-4b6e-bcd9-4626d55daaa2-workload-socket\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Jul 21 13:08:59.524480 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:08:59.524456    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3ce5ffbf-b61f-4b6e-bcd9-4626d55daaa2-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "3ce5ffbf-b61f-4b6e-bcd9-4626d55daaa2" (UID: "3ce5ffbf-b61f-4b6e-bcd9-4626d55daaa2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 21 13:08:59.524667 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:08:59.524649    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3ce5ffbf-b61f-4b6e-bcd9-4626d55daaa2-istio-token" (OuterVolumeSpecName: "istio-token") pod "3ce5ffbf-b61f-4b6e-bcd9-4626d55daaa2" (UID: "3ce5ffbf-b61f-4b6e-bcd9-4626d55daaa2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 13:08:59.524724 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:08:59.524648    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3ce5ffbf-b61f-4b6e-bcd9-4626d55daaa2-kube-api-access-bsjqh" (OuterVolumeSpecName: "kube-api-access-bsjqh") pod "3ce5ffbf-b61f-4b6e-bcd9-4626d55daaa2" (UID: "3ce5ffbf-b61f-4b6e-bcd9-4626d55daaa2"). InnerVolumeSpecName "kube-api-access-bsjqh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 13:08:59.524762 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:08:59.524724    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ce5ffbf-b61f-4b6e-bcd9-4626d55daaa2-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "3ce5ffbf-b61f-4b6e-bcd9-4626d55daaa2" (UID: "3ce5ffbf-b61f-4b6e-bcd9-4626d55daaa2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 13:08:59.623823 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:08:59.623770    2556 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3ce5ffbf-b61f-4b6e-bcd9-4626d55daaa2-istio-data\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Jul 21 13:08:59.623823 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:08:59.623791    2556 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3ce5ffbf-b61f-4b6e-bcd9-4626d55daaa2-istio-podinfo\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Jul 21 13:08:59.623823 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:08:59.623801    2556 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ce5ffbf-b61f-4b6e-bcd9-4626d55daaa2-credential-socket\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Jul 21 13:08:59.623823 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:08:59.623808    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bsjqh\" (UniqueName: \"kubernetes.io/projected/3ce5ffbf-b61f-4b6e-bcd9-4626d55daaa2-kube-api-access-bsjqh\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Jul 21 13:08:59.623823 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:08:59.623817    2556 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3ce5ffbf-b61f-4b6e-bcd9-4626d55daaa2-istio-envoy\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Jul 21 13:08:59.623823 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:08:59.623824    2556 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3ce5ffbf-b61f-4b6e-bcd9-4626d55daaa2-istio-token\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Jul 21 13:09:00.180254 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:09:00.180217    2556 generic.go:356] "Generic (PLEG): container finished" podID="3ce5ffbf-b61f-4b6e-bcd9-4626d55daaa2" containerID="4c3fae2a6f8c3672ca0cabc6441d4adf2cec5093468d93ab65cf16facf80ed3d" exitCode=0
Jul 21 13:09:00.180413 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:09:00.180272    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e773b8a4-openshift-default-6455b8c6c7-95ssv" event={"ID":"3ce5ffbf-b61f-4b6e-bcd9-4626d55daaa2","Type":"ContainerDied","Data":"4c3fae2a6f8c3672ca0cabc6441d4adf2cec5093468d93ab65cf16facf80ed3d"}
Jul 21 13:09:00.180413 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:09:00.180279    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-e773b8a4-openshift-default-6455b8c6c7-95ssv"
Jul 21 13:09:00.180413 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:09:00.180337    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e773b8a4-openshift-default-6455b8c6c7-95ssv" event={"ID":"3ce5ffbf-b61f-4b6e-bcd9-4626d55daaa2","Type":"ContainerDied","Data":"9476917ec4b795ebce360743c9f2f23fe226ad8ec2319f79bba635e9c029296c"}
Jul 21 13:09:00.180413 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:09:00.180359    2556 scope.go:117] "RemoveContainer" containerID="4c3fae2a6f8c3672ca0cabc6441d4adf2cec5093468d93ab65cf16facf80ed3d"
Jul 21 13:09:00.193744 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:09:00.193726    2556 scope.go:117] "RemoveContainer" containerID="a9d3b724646bb1b7561e4e5f69ad2704f88f792132101e06d57a503c5b6d31b3"
Jul 21 13:09:00.202282 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:09:00.202260    2556 scope.go:117] "RemoveContainer" containerID="4c3fae2a6f8c3672ca0cabc6441d4adf2cec5093468d93ab65cf16facf80ed3d"
Jul 21 13:09:00.202592 ip-10-0-130-2 kubenswrapper[2556]: E0721 13:09:00.202574    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4c3fae2a6f8c3672ca0cabc6441d4adf2cec5093468d93ab65cf16facf80ed3d\": container with ID starting with 4c3fae2a6f8c3672ca0cabc6441d4adf2cec5093468d93ab65cf16facf80ed3d not found: ID does not exist" containerID="4c3fae2a6f8c3672ca0cabc6441d4adf2cec5093468d93ab65cf16facf80ed3d"
Jul 21 13:09:00.202655 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:09:00.202601    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4c3fae2a6f8c3672ca0cabc6441d4adf2cec5093468d93ab65cf16facf80ed3d"} err="failed to get container status \"4c3fae2a6f8c3672ca0cabc6441d4adf2cec5093468d93ab65cf16facf80ed3d\": rpc error: code = NotFound desc = could not find container \"4c3fae2a6f8c3672ca0cabc6441d4adf2cec5093468d93ab65cf16facf80ed3d\": container with ID starting with 4c3fae2a6f8c3672ca0cabc6441d4adf2cec5093468d93ab65cf16facf80ed3d not found: ID does not exist"
Jul 21 13:09:00.202655 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:09:00.202618    2556 scope.go:117] "RemoveContainer" containerID="a9d3b724646bb1b7561e4e5f69ad2704f88f792132101e06d57a503c5b6d31b3"
Jul 21 13:09:00.202885 ip-10-0-130-2 kubenswrapper[2556]: E0721 13:09:00.202867    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a9d3b724646bb1b7561e4e5f69ad2704f88f792132101e06d57a503c5b6d31b3\": container with ID starting with a9d3b724646bb1b7561e4e5f69ad2704f88f792132101e06d57a503c5b6d31b3 not found: ID does not exist" containerID="a9d3b724646bb1b7561e4e5f69ad2704f88f792132101e06d57a503c5b6d31b3"
Jul 21 13:09:00.202946 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:09:00.202894    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a9d3b724646bb1b7561e4e5f69ad2704f88f792132101e06d57a503c5b6d31b3"} err="failed to get container status \"a9d3b724646bb1b7561e4e5f69ad2704f88f792132101e06d57a503c5b6d31b3\": rpc error: code = NotFound desc = could not find container \"a9d3b724646bb1b7561e4e5f69ad2704f88f792132101e06d57a503c5b6d31b3\": container with ID starting with a9d3b724646bb1b7561e4e5f69ad2704f88f792132101e06d57a503c5b6d31b3 not found: ID does not exist"
Jul 21 13:09:00.204246 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:09:00.204227    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-e773b8a4-openshift-default-6455b8c6c7-95ssv"]
Jul 21 13:09:00.209240 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:09:00.209219    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-e773b8a4-openshift-default-6455b8c6c7-95ssv"]
Jul 21 13:09:00.561702 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:09:00.561677    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-f454aa60-64f8b9d958-b6c6v"]
Jul 21 13:09:00.562129 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:09:00.562115    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3ce5ffbf-b61f-4b6e-bcd9-4626d55daaa2" containerName="istio-proxy"
Jul 21 13:09:00.562186 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:09:00.562131    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="3ce5ffbf-b61f-4b6e-bcd9-4626d55daaa2" containerName="istio-proxy"
Jul 21 13:09:00.562186 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:09:00.562149    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3ce5ffbf-b61f-4b6e-bcd9-4626d55daaa2" containerName="istio-proxy"
Jul 21 13:09:00.562186 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:09:00.562154    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="3ce5ffbf-b61f-4b6e-bcd9-4626d55daaa2" containerName="istio-proxy"
Jul 21 13:09:00.562186 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:09:00.562161    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="05db79fe-7e83-45c4-96d3-e68f6dcc000e" containerName="revoke-keys"
Jul 21 13:09:00.562186 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:09:00.562167    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="05db79fe-7e83-45c4-96d3-e68f6dcc000e" containerName="revoke-keys"
Jul 21 13:09:00.562363 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:09:00.562231    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="3ce5ffbf-b61f-4b6e-bcd9-4626d55daaa2" containerName="istio-proxy"
Jul 21 13:09:00.562363 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:09:00.562242    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="05db79fe-7e83-45c4-96d3-e68f6dcc000e" containerName="revoke-keys"
Jul 21 13:09:00.566898 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:09:00.566881    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-f454aa60-64f8b9d958-b6c6v"
Jul 21 13:09:00.572492 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:09:00.572471    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-f454aa60-serving-cert\""
Jul 21 13:09:00.578541 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:09:00.578498    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-f454aa60-64f8b9d958-b6c6v"]
Jul 21 13:09:00.731603 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:09:00.731573    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ssgs9\" (UniqueName: \"kubernetes.io/projected/fe45f321-478c-4398-a8d6-77a20dca80a6-kube-api-access-ssgs9\") pod \"maas-api-e2e-migrate-f454aa60-64f8b9d958-b6c6v\" (UID: \"fe45f321-478c-4398-a8d6-77a20dca80a6\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-f454aa60-64f8b9d958-b6c6v"
Jul 21 13:09:00.731748 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:09:00.731609    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fe45f321-478c-4398-a8d6-77a20dca80a6-maas-api-tls\") pod \"maas-api-e2e-migrate-f454aa60-64f8b9d958-b6c6v\" (UID: \"fe45f321-478c-4398-a8d6-77a20dca80a6\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-f454aa60-64f8b9d958-b6c6v"
Jul 21 13:09:00.731748 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:09:00.731639    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fe45f321-478c-4398-a8d6-77a20dca80a6-openshift-service-ca\") pod \"maas-api-e2e-migrate-f454aa60-64f8b9d958-b6c6v\" (UID: \"fe45f321-478c-4398-a8d6-77a20dca80a6\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-f454aa60-64f8b9d958-b6c6v"
Jul 21 13:09:00.833068 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:09:00.832980    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ssgs9\" (UniqueName: \"kubernetes.io/projected/fe45f321-478c-4398-a8d6-77a20dca80a6-kube-api-access-ssgs9\") pod \"maas-api-e2e-migrate-f454aa60-64f8b9d958-b6c6v\" (UID: \"fe45f321-478c-4398-a8d6-77a20dca80a6\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-f454aa60-64f8b9d958-b6c6v"
Jul 21 13:09:00.833068 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:09:00.833037    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fe45f321-478c-4398-a8d6-77a20dca80a6-maas-api-tls\") pod \"maas-api-e2e-migrate-f454aa60-64f8b9d958-b6c6v\" (UID: \"fe45f321-478c-4398-a8d6-77a20dca80a6\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-f454aa60-64f8b9d958-b6c6v"
Jul 21 13:09:00.833234 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:09:00.833069    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fe45f321-478c-4398-a8d6-77a20dca80a6-openshift-service-ca\") pod \"maas-api-e2e-migrate-f454aa60-64f8b9d958-b6c6v\" (UID: \"fe45f321-478c-4398-a8d6-77a20dca80a6\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-f454aa60-64f8b9d958-b6c6v"
Jul 21 13:09:00.833722 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:09:00.833704    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fe45f321-478c-4398-a8d6-77a20dca80a6-openshift-service-ca\") pod \"maas-api-e2e-migrate-f454aa60-64f8b9d958-b6c6v\" (UID: \"fe45f321-478c-4398-a8d6-77a20dca80a6\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-f454aa60-64f8b9d958-b6c6v"
Jul 21 13:09:00.835735 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:09:00.835712    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fe45f321-478c-4398-a8d6-77a20dca80a6-maas-api-tls\") pod \"maas-api-e2e-migrate-f454aa60-64f8b9d958-b6c6v\" (UID: \"fe45f321-478c-4398-a8d6-77a20dca80a6\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-f454aa60-64f8b9d958-b6c6v"
Jul 21 13:09:00.842432 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:09:00.842414    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ssgs9\" (UniqueName: \"kubernetes.io/projected/fe45f321-478c-4398-a8d6-77a20dca80a6-kube-api-access-ssgs9\") pod \"maas-api-e2e-migrate-f454aa60-64f8b9d958-b6c6v\" (UID: \"fe45f321-478c-4398-a8d6-77a20dca80a6\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-f454aa60-64f8b9d958-b6c6v"
Jul 21 13:09:00.877282 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:09:00.877262    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-f454aa60-64f8b9d958-b6c6v"
Jul 21 13:09:01.014959 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:09:01.014928    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-f454aa60-64f8b9d958-b6c6v"]
Jul 21 13:09:01.016714 ip-10-0-130-2 kubenswrapper[2556]: W0721 13:09:01.016688    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe45f321_478c_4398_a8d6_77a20dca80a6.slice/crio-a9ce5605c588e972a719fc8ae333e04a54b9ae1fdbbdb7f9a81132137567dedb WatchSource:0}: Error finding container a9ce5605c588e972a719fc8ae333e04a54b9ae1fdbbdb7f9a81132137567dedb: Status 404 returned error can't find the container with id a9ce5605c588e972a719fc8ae333e04a54b9ae1fdbbdb7f9a81132137567dedb
Jul 21 13:09:01.018450 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:09:01.018432    2556 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 21 13:09:01.187715 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:09:01.187642    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-f454aa60-64f8b9d958-b6c6v" event={"ID":"fe45f321-478c-4398-a8d6-77a20dca80a6","Type":"ContainerStarted","Data":"a9ce5605c588e972a719fc8ae333e04a54b9ae1fdbbdb7f9a81132137567dedb"}
Jul 21 13:09:01.194867 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:09:01.194833    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3ce5ffbf-b61f-4b6e-bcd9-4626d55daaa2" path="/var/lib/kubelet/pods/3ce5ffbf-b61f-4b6e-bcd9-4626d55daaa2/volumes"
Jul 21 13:09:02.194508 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:09:02.194471    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-f454aa60-64f8b9d958-b6c6v" event={"ID":"fe45f321-478c-4398-a8d6-77a20dca80a6","Type":"ContainerStarted","Data":"b6d0cc59ad009b6433151cbd519ec4d00c87ed4d66a8620e245a8c25c90bbd86"}
Jul 21 13:09:02.194879 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:09:02.194542    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-f454aa60-64f8b9d958-b6c6v"
Jul 21 13:09:02.214134 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:09:02.214088    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-f454aa60-64f8b9d958-b6c6v" podStartSLOduration=1.858842823 podStartE2EDuration="2.214072719s" totalImagesPullingTime="355.229896ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 13:09:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:09:02.212337373 +0000 UTC m=+1755.516342464" watchObservedRunningTime="2026-07-21 13:09:02.214072719 +0000 UTC m=+1755.518077810"
Jul 21 13:09:08.205806 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:09:08.205773    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-f454aa60-64f8b9d958-b6c6v"
Jul 21 13:09:25.442114 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:09:25.442081    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f454aa60-5q7vs"]
Jul 21 13:09:25.442575 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:09:25.442561    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="3ce5ffbf-b61f-4b6e-bcd9-4626d55daaa2" containerName="istio-proxy"
Jul 21 13:09:25.445686 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:09:25.445660    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f454aa60-5q7vs"
Jul 21 13:09:25.447923 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:09:25.447904    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5nsrr\""
Jul 21 13:09:25.453712 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:09:25.453689    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f454aa60-5q7vs"]
Jul 21 13:09:25.538289 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:09:25.538261    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/240acb44-9542-4ffc-8d77-9a73cc79300d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-f454aa60-5q7vs\" (UID: \"240acb44-9542-4ffc-8d77-9a73cc79300d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f454aa60-5q7vs"
Jul 21 13:09:25.639006 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:09:25.638975    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/240acb44-9542-4ffc-8d77-9a73cc79300d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-f454aa60-5q7vs\" (UID: \"240acb44-9542-4ffc-8d77-9a73cc79300d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f454aa60-5q7vs"
Jul 21 13:09:25.639567 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:09:25.639547    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/240acb44-9542-4ffc-8d77-9a73cc79300d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-f454aa60-5q7vs\" (UID: \"240acb44-9542-4ffc-8d77-9a73cc79300d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f454aa60-5q7vs"
Jul 21 13:09:25.759130 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:09:25.759110    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f454aa60-5q7vs"
Jul 21 13:09:25.894375 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:09:25.894349    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f454aa60-5q7vs"]
Jul 21 13:09:25.896559 ip-10-0-130-2 kubenswrapper[2556]: W0721 13:09:25.896534    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod240acb44_9542_4ffc_8d77_9a73cc79300d.slice/crio-c76c0a17b35eec97b599f3cb25ccba87308ace89c8f57821c63b5f48fdea15b8 WatchSource:0}: Error finding container c76c0a17b35eec97b599f3cb25ccba87308ace89c8f57821c63b5f48fdea15b8: Status 404 returned error can't find the container with id c76c0a17b35eec97b599f3cb25ccba87308ace89c8f57821c63b5f48fdea15b8
Jul 21 13:09:26.337729 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:09:26.337698    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f454aa60-5q7vs" event={"ID":"240acb44-9542-4ffc-8d77-9a73cc79300d","Type":"ContainerStarted","Data":"6db47655fe1d5b014f632cd59f6913c36585a5072d27b67ca70e155e3e03a684"}
Jul 21 13:09:26.337918 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:09:26.337750    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f454aa60-5q7vs" event={"ID":"240acb44-9542-4ffc-8d77-9a73cc79300d","Type":"ContainerStarted","Data":"c76c0a17b35eec97b599f3cb25ccba87308ace89c8f57821c63b5f48fdea15b8"}
Jul 21 13:09:26.355718 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:09:26.355622    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f454aa60-5q7vs" podStartSLOduration=1.355607229 podStartE2EDuration="1.355607229s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 13:09:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:09:26.353260443 +0000 UTC m=+1779.657265556" watchObservedRunningTime="2026-07-21 13:09:26.355607229 +0000 UTC m=+1779.659612318"
Jul 21 13:09:27.344796 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:09:27.344760    2556 generic.go:356] "Generic (PLEG): container finished" podID="240acb44-9542-4ffc-8d77-9a73cc79300d" containerID="6db47655fe1d5b014f632cd59f6913c36585a5072d27b67ca70e155e3e03a684" exitCode=0
Jul 21 13:09:27.345250 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:09:27.344794    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f454aa60-5q7vs" event={"ID":"240acb44-9542-4ffc-8d77-9a73cc79300d","Type":"ContainerDied","Data":"6db47655fe1d5b014f632cd59f6913c36585a5072d27b67ca70e155e3e03a684"}
Jul 21 13:09:28.495178 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:09:28.495155    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f454aa60-5q7vs"
Jul 21 13:09:28.566238 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:09:28.566213    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/240acb44-9542-4ffc-8d77-9a73cc79300d-maas-api-service-ca\") pod \"240acb44-9542-4ffc-8d77-9a73cc79300d\" (UID: \"240acb44-9542-4ffc-8d77-9a73cc79300d\") "
Jul 21 13:09:28.566502 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:09:28.566481    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/240acb44-9542-4ffc-8d77-9a73cc79300d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "240acb44-9542-4ffc-8d77-9a73cc79300d" (UID: "240acb44-9542-4ffc-8d77-9a73cc79300d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 13:09:28.666986 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:09:28.666923    2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/240acb44-9542-4ffc-8d77-9a73cc79300d-maas-api-service-ca\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Jul 21 13:09:29.358627 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:09:29.358600    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f454aa60-5q7vs"
Jul 21 13:09:29.358790 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:09:29.358637    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f454aa60-5q7vs" event={"ID":"240acb44-9542-4ffc-8d77-9a73cc79300d","Type":"ContainerDied","Data":"c76c0a17b35eec97b599f3cb25ccba87308ace89c8f57821c63b5f48fdea15b8"}
Jul 21 13:09:29.358790 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:09:29.358688    2556 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c76c0a17b35eec97b599f3cb25ccba87308ace89c8f57821c63b5f48fdea15b8"
Jul 21 13:09:36.183212 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:09:36.183180    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-f454aa60-64f8b9d958-b6c6v"]
Jul 21 13:09:36.183599 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:09:36.183365    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-f454aa60-64f8b9d958-b6c6v" podUID="fe45f321-478c-4398-a8d6-77a20dca80a6" containerName="maas-api" containerID="cri-o://b6d0cc59ad009b6433151cbd519ec4d00c87ed4d66a8620e245a8c25c90bbd86" gracePeriod=30
Jul 21 13:09:36.398679 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:09:36.398647    2556 generic.go:356] "Generic (PLEG): container finished" podID="fe45f321-478c-4398-a8d6-77a20dca80a6" containerID="b6d0cc59ad009b6433151cbd519ec4d00c87ed4d66a8620e245a8c25c90bbd86" exitCode=0
Jul 21 13:09:36.398840 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:09:36.398692    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-f454aa60-64f8b9d958-b6c6v" event={"ID":"fe45f321-478c-4398-a8d6-77a20dca80a6","Type":"ContainerDied","Data":"b6d0cc59ad009b6433151cbd519ec4d00c87ed4d66a8620e245a8c25c90bbd86"}
Jul 21 13:09:36.451292 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:09:36.451271    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-f454aa60-64f8b9d958-b6c6v"
Jul 21 13:09:36.530881 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:09:36.530856    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fe45f321-478c-4398-a8d6-77a20dca80a6-maas-api-tls\") pod \"fe45f321-478c-4398-a8d6-77a20dca80a6\" (UID: \"fe45f321-478c-4398-a8d6-77a20dca80a6\") "
Jul 21 13:09:36.530999 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:09:36.530893    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fe45f321-478c-4398-a8d6-77a20dca80a6-openshift-service-ca\") pod \"fe45f321-478c-4398-a8d6-77a20dca80a6\" (UID: \"fe45f321-478c-4398-a8d6-77a20dca80a6\") "
Jul 21 13:09:36.530999 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:09:36.530947    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ssgs9\" (UniqueName: \"kubernetes.io/projected/fe45f321-478c-4398-a8d6-77a20dca80a6-kube-api-access-ssgs9\") pod \"fe45f321-478c-4398-a8d6-77a20dca80a6\" (UID: \"fe45f321-478c-4398-a8d6-77a20dca80a6\") "
Jul 21 13:09:36.531301 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:09:36.531278    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fe45f321-478c-4398-a8d6-77a20dca80a6-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "fe45f321-478c-4398-a8d6-77a20dca80a6" (UID: "fe45f321-478c-4398-a8d6-77a20dca80a6"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 13:09:36.533213 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:09:36.533189    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fe45f321-478c-4398-a8d6-77a20dca80a6-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "fe45f321-478c-4398-a8d6-77a20dca80a6" (UID: "fe45f321-478c-4398-a8d6-77a20dca80a6"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 13:09:36.533292 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:09:36.533274    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fe45f321-478c-4398-a8d6-77a20dca80a6-kube-api-access-ssgs9" (OuterVolumeSpecName: "kube-api-access-ssgs9") pod "fe45f321-478c-4398-a8d6-77a20dca80a6" (UID: "fe45f321-478c-4398-a8d6-77a20dca80a6"). InnerVolumeSpecName "kube-api-access-ssgs9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 13:09:36.632277 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:09:36.632256    2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fe45f321-478c-4398-a8d6-77a20dca80a6-maas-api-tls\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Jul 21 13:09:36.632277 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:09:36.632275    2556 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fe45f321-478c-4398-a8d6-77a20dca80a6-openshift-service-ca\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Jul 21 13:09:36.632416 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:09:36.632285    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ssgs9\" (UniqueName: \"kubernetes.io/projected/fe45f321-478c-4398-a8d6-77a20dca80a6-kube-api-access-ssgs9\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Jul 21 13:09:37.404304 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:09:37.404284    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-f454aa60-64f8b9d958-b6c6v"
Jul 21 13:09:37.404642 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:09:37.404330    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-f454aa60-64f8b9d958-b6c6v" event={"ID":"fe45f321-478c-4398-a8d6-77a20dca80a6","Type":"ContainerDied","Data":"a9ce5605c588e972a719fc8ae333e04a54b9ae1fdbbdb7f9a81132137567dedb"}
Jul 21 13:09:37.404642 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:09:37.404373    2556 scope.go:117] "RemoveContainer" containerID="b6d0cc59ad009b6433151cbd519ec4d00c87ed4d66a8620e245a8c25c90bbd86"
Jul 21 13:09:37.422959 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:09:37.422933    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-f454aa60-64f8b9d958-b6c6v"]
Jul 21 13:09:37.427805 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:09:37.427783    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-f454aa60-64f8b9d958-b6c6v"]
Jul 21 13:09:39.194222 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:09:39.194152    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fe45f321-478c-4398-a8d6-77a20dca80a6" path="/var/lib/kubelet/pods/fe45f321-478c-4398-a8d6-77a20dca80a6/volumes"
Jul 21 13:09:40.549003 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:09:40.548964    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f454aa60-5q7vs"]
Jul 21 13:09:40.552494 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:09:40.552468    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f454aa60-5q7vs"]
Jul 21 13:09:41.194099 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:09:41.194069    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="240acb44-9542-4ffc-8d77-9a73cc79300d" path="/var/lib/kubelet/pods/240acb44-9542-4ffc-8d77-9a73cc79300d/volumes"
Jul 21 13:09:41.478365 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:09:41.478338    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-1f53c0d5-openshift-default-cc8f47c49-m84lh"]
Jul 21 13:09:41.478755 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:09:41.478743    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fe45f321-478c-4398-a8d6-77a20dca80a6" containerName="maas-api"
Jul 21 13:09:41.478798 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:09:41.478757    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="fe45f321-478c-4398-a8d6-77a20dca80a6" containerName="maas-api"
Jul 21 13:09:41.478798 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:09:41.478769    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="240acb44-9542-4ffc-8d77-9a73cc79300d" containerName="revoke-keys"
Jul 21 13:09:41.478798 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:09:41.478774    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="240acb44-9542-4ffc-8d77-9a73cc79300d" containerName="revoke-keys"
Jul 21 13:09:41.478888 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:09:41.478840    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="240acb44-9542-4ffc-8d77-9a73cc79300d" containerName="revoke-keys"
Jul 21 13:09:41.478888 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:09:41.478852    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="fe45f321-478c-4398-a8d6-77a20dca80a6" containerName="maas-api"
Jul 21 13:09:41.483383 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:09:41.483364    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-1f53c0d5-openshift-default-cc8f47c49-m84lh"
Jul 21 13:09:41.490349 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:09:41.490131    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-1f53c0d5-openshift-default-dockercfg-rqhn7\""
Jul 21 13:09:41.497257 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:09:41.497235    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-1f53c0d5-openshift-default-cc8f47c49-m84lh"]
Jul 21 13:09:41.573806 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:09:41.573770    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a781e1f8-c90e-4825-bbf0-3b484128736b-istio-data\") pod \"e2e-ait-1f53c0d5-openshift-default-cc8f47c49-m84lh\" (UID: \"a781e1f8-c90e-4825-bbf0-3b484128736b\") " pod="openshift-ingress/e2e-ait-1f53c0d5-openshift-default-cc8f47c49-m84lh"
Jul 21 13:09:41.574399 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:09:41.573858    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a781e1f8-c90e-4825-bbf0-3b484128736b-credential-socket\") pod \"e2e-ait-1f53c0d5-openshift-default-cc8f47c49-m84lh\" (UID: \"a781e1f8-c90e-4825-bbf0-3b484128736b\") " pod="openshift-ingress/e2e-ait-1f53c0d5-openshift-default-cc8f47c49-m84lh"
Jul 21 13:09:41.574399 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:09:41.573913    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a781e1f8-c90e-4825-bbf0-3b484128736b-istiod-ca-cert\") pod \"e2e-ait-1f53c0d5-openshift-default-cc8f47c49-m84lh\" (UID: \"a781e1f8-c90e-4825-bbf0-3b484128736b\") " pod="openshift-ingress/e2e-ait-1f53c0d5-openshift-default-cc8f47c49-m84lh"
Jul 21 13:09:41.574399 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:09:41.573965    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a781e1f8-c90e-4825-bbf0-3b484128736b-istio-token\") pod \"e2e-ait-1f53c0d5-openshift-default-cc8f47c49-m84lh\" (UID: \"a781e1f8-c90e-4825-bbf0-3b484128736b\") " pod="openshift-ingress/e2e-ait-1f53c0d5-openshift-default-cc8f47c49-m84lh"
Jul 21 13:09:41.574399 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:09:41.573996    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kbmrf\" (UniqueName: \"kubernetes.io/projected/a781e1f8-c90e-4825-bbf0-3b484128736b-kube-api-access-kbmrf\") pod \"e2e-ait-1f53c0d5-openshift-default-cc8f47c49-m84lh\" (UID: \"a781e1f8-c90e-4825-bbf0-3b484128736b\") " pod="openshift-ingress/e2e-ait-1f53c0d5-openshift-default-cc8f47c49-m84lh"
Jul 21 13:09:41.574399 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:09:41.574078    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a781e1f8-c90e-4825-bbf0-3b484128736b-istio-envoy\") pod \"e2e-ait-1f53c0d5-openshift-default-cc8f47c49-m84lh\" (UID: \"a781e1f8-c90e-4825-bbf0-3b484128736b\") " pod="openshift-ingress/e2e-ait-1f53c0d5-openshift-default-cc8f47c49-m84lh"
Jul 21 13:09:41.574399 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:09:41.574127    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a781e1f8-c90e-4825-bbf0-3b484128736b-workload-certs\") pod \"e2e-ait-1f53c0d5-openshift-default-cc8f47c49-m84lh\" (UID: \"a781e1f8-c90e-4825-bbf0-3b484128736b\") " pod="openshift-ingress/e2e-ait-1f53c0d5-openshift-default-cc8f47c49-m84lh"
Jul 21 13:09:41.574399 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:09:41.574166    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a781e1f8-c90e-4825-bbf0-3b484128736b-workload-socket\") pod \"e2e-ait-1f53c0d5-openshift-default-cc8f47c49-m84lh\" (UID: \"a781e1f8-c90e-4825-bbf0-3b484128736b\") " pod="openshift-ingress/e2e-ait-1f53c0d5-openshift-default-cc8f47c49-m84lh"
Jul 21 13:09:41.574399 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:09:41.574243    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a781e1f8-c90e-4825-bbf0-3b484128736b-istio-podinfo\") pod \"e2e-ait-1f53c0d5-openshift-default-cc8f47c49-m84lh\" (UID: \"a781e1f8-c90e-4825-bbf0-3b484128736b\") " pod="openshift-ingress/e2e-ait-1f53c0d5-openshift-default-cc8f47c49-m84lh"
Jul 21 13:09:41.674664 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:09:41.674634    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a781e1f8-c90e-4825-bbf0-3b484128736b-credential-socket\") pod \"e2e-ait-1f53c0d5-openshift-default-cc8f47c49-m84lh\" (UID: \"a781e1f8-c90e-4825-bbf0-3b484128736b\") " pod="openshift-ingress/e2e-ait-1f53c0d5-openshift-default-cc8f47c49-m84lh"
Jul 21 13:09:41.674820 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:09:41.674670    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a781e1f8-c90e-4825-bbf0-3b484128736b-istiod-ca-cert\") pod \"e2e-ait-1f53c0d5-openshift-default-cc8f47c49-m84lh\" (UID: \"a781e1f8-c90e-4825-bbf0-3b484128736b\") " pod="openshift-ingress/e2e-ait-1f53c0d5-openshift-default-cc8f47c49-m84lh"
Jul 21 13:09:41.674820 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:09:41.674696    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a781e1f8-c90e-4825-bbf0-3b484128736b-istio-token\") pod \"e2e-ait-1f53c0d5-openshift-default-cc8f47c49-m84lh\" (UID: \"a781e1f8-c90e-4825-bbf0-3b484128736b\") " pod="openshift-ingress/e2e-ait-1f53c0d5-openshift-default-cc8f47c49-m84lh"
Jul 21 13:09:41.674820 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:09:41.674713    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kbmrf\" (UniqueName: \"kubernetes.io/projected/a781e1f8-c90e-4825-bbf0-3b484128736b-kube-api-access-kbmrf\") pod \"e2e-ait-1f53c0d5-openshift-default-cc8f47c49-m84lh\" (UID: \"a781e1f8-c90e-4825-bbf0-3b484128736b\") " pod="openshift-ingress/e2e-ait-1f53c0d5-openshift-default-cc8f47c49-m84lh"
Jul 21 13:09:41.674820 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:09:41.674744    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a781e1f8-c90e-4825-bbf0-3b484128736b-istio-envoy\") pod \"e2e-ait-1f53c0d5-openshift-default-cc8f47c49-m84lh\" (UID: \"a781e1f8-c90e-4825-bbf0-3b484128736b\") " pod="openshift-ingress/e2e-ait-1f53c0d5-openshift-default-cc8f47c49-m84lh"
Jul 21 13:09:41.674820 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:09:41.674785    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a781e1f8-c90e-4825-bbf0-3b484128736b-workload-certs\") pod \"e2e-ait-1f53c0d5-openshift-default-cc8f47c49-m84lh\" (UID: \"a781e1f8-c90e-4825-bbf0-3b484128736b\") " pod="openshift-ingress/e2e-ait-1f53c0d5-openshift-default-cc8f47c49-m84lh"
Jul 21 13:09:41.674820 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:09:41.674817    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a781e1f8-c90e-4825-bbf0-3b484128736b-workload-socket\") pod \"e2e-ait-1f53c0d5-openshift-default-cc8f47c49-m84lh\" (UID: \"a781e1f8-c90e-4825-bbf0-3b484128736b\") " pod="openshift-ingress/e2e-ait-1f53c0d5-openshift-default-cc8f47c49-m84lh"
Jul 21 13:09:41.675263 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:09:41.674852    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a781e1f8-c90e-4825-bbf0-3b484128736b-istio-podinfo\") pod \"e2e-ait-1f53c0d5-openshift-default-cc8f47c49-m84lh\" (UID: \"a781e1f8-c90e-4825-bbf0-3b484128736b\") " pod="openshift-ingress/e2e-ait-1f53c0d5-openshift-default-cc8f47c49-m84lh"
Jul 21 13:09:41.675263 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:09:41.674918    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a781e1f8-c90e-4825-bbf0-3b484128736b-istio-data\") pod \"e2e-ait-1f53c0d5-openshift-default-cc8f47c49-m84lh\" (UID: \"a781e1f8-c90e-4825-bbf0-3b484128736b\") " pod="openshift-ingress/e2e-ait-1f53c0d5-openshift-default-cc8f47c49-m84lh"
Jul 21 13:09:41.675263 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:09:41.675085    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a781e1f8-c90e-4825-bbf0-3b484128736b-credential-socket\") pod \"e2e-ait-1f53c0d5-openshift-default-cc8f47c49-m84lh\" (UID: \"a781e1f8-c90e-4825-bbf0-3b484128736b\") " pod="openshift-ingress/e2e-ait-1f53c0d5-openshift-default-cc8f47c49-m84lh"
Jul 21 13:09:41.675263 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:09:41.675158    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a781e1f8-c90e-4825-bbf0-3b484128736b-workload-certs\") pod \"e2e-ait-1f53c0d5-openshift-default-cc8f47c49-m84lh\" (UID: \"a781e1f8-c90e-4825-bbf0-3b484128736b\") " pod="openshift-ingress/e2e-ait-1f53c0d5-openshift-default-cc8f47c49-m84lh"
Jul 21 13:09:41.675263 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:09:41.675233    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a781e1f8-c90e-4825-bbf0-3b484128736b-workload-socket\") pod \"e2e-ait-1f53c0d5-openshift-default-cc8f47c49-m84lh\" (UID: \"a781e1f8-c90e-4825-bbf0-3b484128736b\") " pod="openshift-ingress/e2e-ait-1f53c0d5-openshift-default-cc8f47c49-m84lh"
Jul 21 13:09:41.675527 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:09:41.675283    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a781e1f8-c90e-4825-bbf0-3b484128736b-istio-data\") pod \"e2e-ait-1f53c0d5-openshift-default-cc8f47c49-m84lh\" (UID: \"a781e1f8-c90e-4825-bbf0-3b484128736b\") " pod="openshift-ingress/e2e-ait-1f53c0d5-openshift-default-cc8f47c49-m84lh"
Jul 21 13:09:41.675722 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:09:41.675701    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a781e1f8-c90e-4825-bbf0-3b484128736b-istiod-ca-cert\") pod \"e2e-ait-1f53c0d5-openshift-default-cc8f47c49-m84lh\" (UID: \"a781e1f8-c90e-4825-bbf0-3b484128736b\") " pod="openshift-ingress/e2e-ait-1f53c0d5-openshift-default-cc8f47c49-m84lh"
Jul 21 13:09:41.677087 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:09:41.677064    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a781e1f8-c90e-4825-bbf0-3b484128736b-istio-envoy\") pod \"e2e-ait-1f53c0d5-openshift-default-cc8f47c49-m84lh\" (UID: \"a781e1f8-c90e-4825-bbf0-3b484128736b\") " pod="openshift-ingress/e2e-ait-1f53c0d5-openshift-default-cc8f47c49-m84lh"
Jul 21 13:09:41.677752 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:09:41.677729    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a781e1f8-c90e-4825-bbf0-3b484128736b-istio-podinfo\") pod \"e2e-ait-1f53c0d5-openshift-default-cc8f47c49-m84lh\" (UID: \"a781e1f8-c90e-4825-bbf0-3b484128736b\") " pod="openshift-ingress/e2e-ait-1f53c0d5-openshift-default-cc8f47c49-m84lh"
Jul 21 13:09:41.685330 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:09:41.685305    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kbmrf\" (UniqueName: \"kubernetes.io/projected/a781e1f8-c90e-4825-bbf0-3b484128736b-kube-api-access-kbmrf\") pod \"e2e-ait-1f53c0d5-openshift-default-cc8f47c49-m84lh\" (UID: \"a781e1f8-c90e-4825-bbf0-3b484128736b\") " pod="openshift-ingress/e2e-ait-1f53c0d5-openshift-default-cc8f47c49-m84lh"
Jul 21 13:09:41.686292 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:09:41.686271    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a781e1f8-c90e-4825-bbf0-3b484128736b-istio-token\") pod \"e2e-ait-1f53c0d5-openshift-default-cc8f47c49-m84lh\" (UID: \"a781e1f8-c90e-4825-bbf0-3b484128736b\") " pod="openshift-ingress/e2e-ait-1f53c0d5-openshift-default-cc8f47c49-m84lh"
Jul 21 13:09:41.793502 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:09:41.793431    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-1f53c0d5-openshift-default-cc8f47c49-m84lh"
Jul 21 13:09:41.941629 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:09:41.941603    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-1f53c0d5-openshift-default-cc8f47c49-m84lh"]
Jul 21 13:09:41.948168 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:09:41.948133    2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 21 13:09:41.948279 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:09:41.948206    2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 21 13:09:41.948279 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:09:41.948237    2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 21 13:09:42.431842 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:09:42.431791    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-1f53c0d5-openshift-default-cc8f47c49-m84lh" event={"ID":"a781e1f8-c90e-4825-bbf0-3b484128736b","Type":"ContainerStarted","Data":"d39f9a91b354dca4f5f6123630d277c3ac646e33045763f50752fccf3a0a82a5"}
Jul 21 13:09:42.432008 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:09:42.431857    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-1f53c0d5-openshift-default-cc8f47c49-m84lh" event={"ID":"a781e1f8-c90e-4825-bbf0-3b484128736b","Type":"ContainerStarted","Data":"f2e3ccddb6570f8bab072d81a9595f293b9b1a5bab0b7ba59be991311a1b7db4"}
Jul 21 13:09:42.459368 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:09:42.459321    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-1f53c0d5-openshift-default-cc8f47c49-m84lh" podStartSLOduration=1.459307459 podStartE2EDuration="1.459307459s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 13:09:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:09:42.458217641 +0000 UTC m=+1795.762222752" watchObservedRunningTime="2026-07-21 13:09:42.459307459 +0000 UTC m=+1795.763312547"
Jul 21 13:09:42.794379 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:09:42.794337    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-1f53c0d5-openshift-default-cc8f47c49-m84lh"
Jul 21 13:09:43.799070 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:09:43.799043    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-1f53c0d5-openshift-default-cc8f47c49-m84lh"
Jul 21 13:09:44.442475 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:09:44.442449    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-1f53c0d5-openshift-default-cc8f47c49-m84lh"
Jul 21 13:09:44.443426 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:09:44.443403    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-1f53c0d5-openshift-default-cc8f47c49-m84lh"
Jul 21 13:09:59.444294 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:09:59.444262    2556 patch_prober.go:28] interesting pod/e2e-ait-1f53c0d5-openshift-default-cc8f47c49-m84lh container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body=
Jul 21 13:09:59.444647 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:09:59.444320    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-1f53c0d5-openshift-default-cc8f47c49-m84lh" podUID="a781e1f8-c90e-4825-bbf0-3b484128736b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused"
Jul 21 13:09:59.521966 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:09:59.521939    2556 generic.go:356] "Generic (PLEG): container finished" podID="a781e1f8-c90e-4825-bbf0-3b484128736b" containerID="d39f9a91b354dca4f5f6123630d277c3ac646e33045763f50752fccf3a0a82a5" exitCode=0
Jul 21 13:09:59.522085 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:09:59.522041    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-1f53c0d5-openshift-default-cc8f47c49-m84lh" event={"ID":"a781e1f8-c90e-4825-bbf0-3b484128736b","Type":"ContainerDied","Data":"d39f9a91b354dca4f5f6123630d277c3ac646e33045763f50752fccf3a0a82a5"}
Jul 21 13:09:59.522382 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:09:59.522369    2556 scope.go:117] "RemoveContainer" containerID="d39f9a91b354dca4f5f6123630d277c3ac646e33045763f50752fccf3a0a82a5"
Jul 21 13:09:59.523341 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:09:59.523314    2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 21 13:09:59.523441 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:09:59.523381    2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 21 13:09:59.523441 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:09:59.523412    2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 21 13:10:00.532779 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:10:00.532750    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-1f53c0d5-openshift-default-cc8f47c49-m84lh" event={"ID":"a781e1f8-c90e-4825-bbf0-3b484128736b","Type":"ContainerStarted","Data":"ef934c9065fe58862f22a8e2a257f0465e17c1f55dd7fdbf5bf2ddd6652eada2"}
Jul 21 13:10:00.793675 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:10:00.793579    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-1f53c0d5-openshift-default-cc8f47c49-m84lh"
Jul 21 13:10:00.793855 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:10:00.793805    2556 patch_prober.go:28] interesting pod/e2e-ait-1f53c0d5-openshift-default-cc8f47c49-m84lh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body=
Jul 21 13:10:00.793942 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:10:00.793877    2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-1f53c0d5-openshift-default-cc8f47c49-m84lh" podUID="a781e1f8-c90e-4825-bbf0-3b484128736b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused"
Jul 21 13:10:01.793833 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:10:01.793780    2556 patch_prober.go:28] interesting pod/e2e-ait-1f53c0d5-openshift-default-cc8f47c49-m84lh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body=
Jul 21 13:10:01.794357 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:10:01.793866    2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-1f53c0d5-openshift-default-cc8f47c49-m84lh" podUID="a781e1f8-c90e-4825-bbf0-3b484128736b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused"
Jul 21 13:10:02.793741 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:10:02.793705    2556 patch_prober.go:28] interesting pod/e2e-ait-1f53c0d5-openshift-default-cc8f47c49-m84lh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body=
Jul 21 13:10:02.794061 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:10:02.793996    2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-1f53c0d5-openshift-default-cc8f47c49-m84lh" podUID="a781e1f8-c90e-4825-bbf0-3b484128736b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused"
Jul 21 13:10:03.794173 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:10:03.794139    2556 patch_prober.go:28] interesting pod/e2e-ait-1f53c0d5-openshift-default-cc8f47c49-m84lh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body=
Jul 21 13:10:03.794795 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:10:03.794730    2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-1f53c0d5-openshift-default-cc8f47c49-m84lh" podUID="a781e1f8-c90e-4825-bbf0-3b484128736b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused"
Jul 21 13:10:04.798051 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:10:04.798002    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-1f53c0d5-openshift-default-cc8f47c49-m84lh"
Jul 21 13:10:04.798433 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:10:04.798302    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-1f53c0d5-openshift-default-cc8f47c49-m84lh"
Jul 21 13:10:04.798919 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:10:04.798901    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-1f53c0d5-openshift-default-cc8f47c49-m84lh"
Jul 21 13:10:09.627867 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:10:09.627831    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-1f53c0d5-vdqxl"]
Jul 21 13:10:09.631664 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:10:09.631648    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-1f53c0d5-vdqxl"
Jul 21 13:10:09.633951 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:10:09.633931    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5nsrr\""
Jul 21 13:10:09.640226 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:10:09.640203    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-1f53c0d5-vdqxl"]
Jul 21 13:10:09.715210 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:10:09.715186    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a97223f7-9972-4345-b6cc-c441c90a39d1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-1f53c0d5-vdqxl\" (UID: \"a97223f7-9972-4345-b6cc-c441c90a39d1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-1f53c0d5-vdqxl"
Jul 21 13:10:09.815900 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:10:09.815875    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a97223f7-9972-4345-b6cc-c441c90a39d1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-1f53c0d5-vdqxl\" (UID: \"a97223f7-9972-4345-b6cc-c441c90a39d1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-1f53c0d5-vdqxl"
Jul 21 13:10:09.816442 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:10:09.816423    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a97223f7-9972-4345-b6cc-c441c90a39d1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-1f53c0d5-vdqxl\" (UID: \"a97223f7-9972-4345-b6cc-c441c90a39d1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-1f53c0d5-vdqxl"
Jul 21 13:10:09.944402 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:10:09.944348    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-1f53c0d5-vdqxl"
Jul 21 13:10:10.287356 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:10:10.287331    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-1f53c0d5-vdqxl"]
Jul 21 13:10:10.289899 ip-10-0-130-2 kubenswrapper[2556]: W0721 13:10:10.289855    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda97223f7_9972_4345_b6cc_c441c90a39d1.slice/crio-df87633a9217033258f33d538713450a0c076cbf5a1d67a2c1743d535f0321fb WatchSource:0}: Error finding container df87633a9217033258f33d538713450a0c076cbf5a1d67a2c1743d535f0321fb: Status 404 returned error can't find the container with id df87633a9217033258f33d538713450a0c076cbf5a1d67a2c1743d535f0321fb
Jul 21 13:10:10.593065 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:10:10.592982    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-1f53c0d5-vdqxl" event={"ID":"a97223f7-9972-4345-b6cc-c441c90a39d1","Type":"ContainerStarted","Data":"7c68a257ac0b4f2dae45645504cf2c2e2a0f7b6b8396714c3dac8f8fc83e7f0f"}
Jul 21 13:10:10.593166 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:10:10.593092    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-1f53c0d5-vdqxl" event={"ID":"a97223f7-9972-4345-b6cc-c441c90a39d1","Type":"ContainerStarted","Data":"df87633a9217033258f33d538713450a0c076cbf5a1d67a2c1743d535f0321fb"}
Jul 21 13:10:10.612224 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:10:10.612172    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-1f53c0d5-vdqxl" podStartSLOduration=1.612158316 podStartE2EDuration="1.612158316s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 13:10:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:10:10.609397644 +0000 UTC m=+1823.913402734" watchObservedRunningTime="2026-07-21 13:10:10.612158316 +0000 UTC m=+1823.916163403"
Jul 21 13:10:11.599505 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:10:11.599475    2556 generic.go:356] "Generic (PLEG): container finished" podID="a97223f7-9972-4345-b6cc-c441c90a39d1" containerID="7c68a257ac0b4f2dae45645504cf2c2e2a0f7b6b8396714c3dac8f8fc83e7f0f" exitCode=0
Jul 21 13:10:11.599820 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:10:11.599554    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-1f53c0d5-vdqxl" event={"ID":"a97223f7-9972-4345-b6cc-c441c90a39d1","Type":"ContainerDied","Data":"7c68a257ac0b4f2dae45645504cf2c2e2a0f7b6b8396714c3dac8f8fc83e7f0f"}
Jul 21 13:10:12.741389 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:10:12.741368    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-1f53c0d5-vdqxl"
Jul 21 13:10:12.844320 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:10:12.844298    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a97223f7-9972-4345-b6cc-c441c90a39d1-maas-api-service-ca\") pod \"a97223f7-9972-4345-b6cc-c441c90a39d1\" (UID: \"a97223f7-9972-4345-b6cc-c441c90a39d1\") "
Jul 21 13:10:12.844631 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:10:12.844608    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a97223f7-9972-4345-b6cc-c441c90a39d1-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a97223f7-9972-4345-b6cc-c441c90a39d1" (UID: "a97223f7-9972-4345-b6cc-c441c90a39d1"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 13:10:12.945279 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:10:12.945224    2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a97223f7-9972-4345-b6cc-c441c90a39d1-maas-api-service-ca\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Jul 21 13:10:13.611895 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:10:13.611870    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-1f53c0d5-vdqxl"
Jul 21 13:10:13.611895 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:10:13.611886    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-1f53c0d5-vdqxl" event={"ID":"a97223f7-9972-4345-b6cc-c441c90a39d1","Type":"ContainerDied","Data":"df87633a9217033258f33d538713450a0c076cbf5a1d67a2c1743d535f0321fb"}
Jul 21 13:10:13.612097 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:10:13.611922    2556 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="df87633a9217033258f33d538713450a0c076cbf5a1d67a2c1743d535f0321fb"
Jul 21 13:10:24.736348 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:10:24.736314    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-1f53c0d5-vdqxl"]
Jul 21 13:10:24.739033 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:10:24.738991    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-1f53c0d5-vdqxl"]
Jul 21 13:10:25.195382 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:10:25.195288    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a97223f7-9972-4345-b6cc-c441c90a39d1" path="/var/lib/kubelet/pods/a97223f7-9972-4345-b6cc-c441c90a39d1/volumes"
Jul 21 13:10:25.918520 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:10:25.918487    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-1f53c0d5-openshift-default-cc8f47c49-m84lh"]
Jul 21 13:10:25.918902 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:10:25.918728    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-1f53c0d5-openshift-default-cc8f47c49-m84lh" podUID="a781e1f8-c90e-4825-bbf0-3b484128736b" containerName="istio-proxy" containerID="cri-o://ef934c9065fe58862f22a8e2a257f0465e17c1f55dd7fdbf5bf2ddd6652eada2" gracePeriod=30
Jul 21 13:10:31.175809 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:10:31.175785    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-1f53c0d5-openshift-default-cc8f47c49-m84lh"
Jul 21 13:10:31.309902 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:10:31.309863    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a781e1f8-c90e-4825-bbf0-3b484128736b-istio-envoy\") pod \"a781e1f8-c90e-4825-bbf0-3b484128736b\" (UID: \"a781e1f8-c90e-4825-bbf0-3b484128736b\") "
Jul 21 13:10:31.309902 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:10:31.309908    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a781e1f8-c90e-4825-bbf0-3b484128736b-istiod-ca-cert\") pod \"a781e1f8-c90e-4825-bbf0-3b484128736b\" (UID: \"a781e1f8-c90e-4825-bbf0-3b484128736b\") "
Jul 21 13:10:31.310156 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:10:31.309955    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a781e1f8-c90e-4825-bbf0-3b484128736b-credential-socket\") pod \"a781e1f8-c90e-4825-bbf0-3b484128736b\" (UID: \"a781e1f8-c90e-4825-bbf0-3b484128736b\") "
Jul 21 13:10:31.310156 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:10:31.309985    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a781e1f8-c90e-4825-bbf0-3b484128736b-istio-podinfo\") pod \"a781e1f8-c90e-4825-bbf0-3b484128736b\" (UID: \"a781e1f8-c90e-4825-bbf0-3b484128736b\") "
Jul 21 13:10:31.310156 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:10:31.310077    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a781e1f8-c90e-4825-bbf0-3b484128736b-workload-socket\") pod \"a781e1f8-c90e-4825-bbf0-3b484128736b\" (UID: \"a781e1f8-c90e-4825-bbf0-3b484128736b\") "
Jul 21 13:10:31.310156 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:10:31.310115    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kbmrf\" (UniqueName: \"kubernetes.io/projected/a781e1f8-c90e-4825-bbf0-3b484128736b-kube-api-access-kbmrf\") pod \"a781e1f8-c90e-4825-bbf0-3b484128736b\" (UID: \"a781e1f8-c90e-4825-bbf0-3b484128736b\") "
Jul 21 13:10:31.310335 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:10:31.310188    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a781e1f8-c90e-4825-bbf0-3b484128736b-workload-certs\") pod \"a781e1f8-c90e-4825-bbf0-3b484128736b\" (UID: \"a781e1f8-c90e-4825-bbf0-3b484128736b\") "
Jul 21 13:10:31.310335 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:10:31.310235    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a781e1f8-c90e-4825-bbf0-3b484128736b-istio-token\") pod \"a781e1f8-c90e-4825-bbf0-3b484128736b\" (UID: \"a781e1f8-c90e-4825-bbf0-3b484128736b\") "
Jul 21 13:10:31.310335 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:10:31.310297    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a781e1f8-c90e-4825-bbf0-3b484128736b-istio-data\") pod \"a781e1f8-c90e-4825-bbf0-3b484128736b\" (UID: \"a781e1f8-c90e-4825-bbf0-3b484128736b\") "
Jul 21 13:10:31.310335 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:10:31.310315    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a781e1f8-c90e-4825-bbf0-3b484128736b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a781e1f8-c90e-4825-bbf0-3b484128736b" (UID: "a781e1f8-c90e-4825-bbf0-3b484128736b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 13:10:31.310531 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:10:31.310360    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a781e1f8-c90e-4825-bbf0-3b484128736b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a781e1f8-c90e-4825-bbf0-3b484128736b" (UID: "a781e1f8-c90e-4825-bbf0-3b484128736b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 13:10:31.310531 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:10:31.310482    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a781e1f8-c90e-4825-bbf0-3b484128736b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a781e1f8-c90e-4825-bbf0-3b484128736b" (UID: "a781e1f8-c90e-4825-bbf0-3b484128736b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 13:10:31.310810 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:10:31.310643    2556 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a781e1f8-c90e-4825-bbf0-3b484128736b-workload-certs\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Jul 21 13:10:31.310810 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:10:31.310662    2556 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a781e1f8-c90e-4825-bbf0-3b484128736b-credential-socket\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Jul 21 13:10:31.310810 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:10:31.310675    2556 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a781e1f8-c90e-4825-bbf0-3b484128736b-workload-socket\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Jul 21 13:10:31.310969 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:10:31.310927    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a781e1f8-c90e-4825-bbf0-3b484128736b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a781e1f8-c90e-4825-bbf0-3b484128736b" (UID: "a781e1f8-c90e-4825-bbf0-3b484128736b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 13:10:31.311573 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:10:31.311544    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a781e1f8-c90e-4825-bbf0-3b484128736b-istio-data" (OuterVolumeSpecName: "istio-data") pod "a781e1f8-c90e-4825-bbf0-3b484128736b" (UID: "a781e1f8-c90e-4825-bbf0-3b484128736b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 13:10:31.312366 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:10:31.312340    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a781e1f8-c90e-4825-bbf0-3b484128736b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a781e1f8-c90e-4825-bbf0-3b484128736b" (UID: "a781e1f8-c90e-4825-bbf0-3b484128736b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 21 13:10:31.312366 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:10:31.312351    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a781e1f8-c90e-4825-bbf0-3b484128736b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a781e1f8-c90e-4825-bbf0-3b484128736b" (UID: "a781e1f8-c90e-4825-bbf0-3b484128736b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 13:10:31.312523 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:10:31.312508    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a781e1f8-c90e-4825-bbf0-3b484128736b-istio-token" (OuterVolumeSpecName: "istio-token") pod "a781e1f8-c90e-4825-bbf0-3b484128736b" (UID: "a781e1f8-c90e-4825-bbf0-3b484128736b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 13:10:31.312808 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:10:31.312791    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a781e1f8-c90e-4825-bbf0-3b484128736b-kube-api-access-kbmrf" (OuterVolumeSpecName: "kube-api-access-kbmrf") pod "a781e1f8-c90e-4825-bbf0-3b484128736b" (UID: "a781e1f8-c90e-4825-bbf0-3b484128736b"). InnerVolumeSpecName "kube-api-access-kbmrf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 13:10:31.411325 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:10:31.411296    2556 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a781e1f8-c90e-4825-bbf0-3b484128736b-istio-token\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Jul 21 13:10:31.411325 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:10:31.411323    2556 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a781e1f8-c90e-4825-bbf0-3b484128736b-istio-data\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Jul 21 13:10:31.411488 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:10:31.411335    2556 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a781e1f8-c90e-4825-bbf0-3b484128736b-istio-envoy\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Jul 21 13:10:31.411488 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:10:31.411360    2556 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a781e1f8-c90e-4825-bbf0-3b484128736b-istiod-ca-cert\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Jul 21 13:10:31.411488 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:10:31.411372    2556 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a781e1f8-c90e-4825-bbf0-3b484128736b-istio-podinfo\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Jul 21 13:10:31.411488 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:10:31.411383    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kbmrf\" (UniqueName: \"kubernetes.io/projected/a781e1f8-c90e-4825-bbf0-3b484128736b-kube-api-access-kbmrf\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Jul 21 13:10:31.715132 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:10:31.715059    2556 generic.go:356] "Generic (PLEG): container finished" podID="a781e1f8-c90e-4825-bbf0-3b484128736b" containerID="ef934c9065fe58862f22a8e2a257f0465e17c1f55dd7fdbf5bf2ddd6652eada2" exitCode=0
Jul 21 13:10:31.715132 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:10:31.715104    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-1f53c0d5-openshift-default-cc8f47c49-m84lh"
Jul 21 13:10:31.715132 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:10:31.715112    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-1f53c0d5-openshift-default-cc8f47c49-m84lh" event={"ID":"a781e1f8-c90e-4825-bbf0-3b484128736b","Type":"ContainerDied","Data":"ef934c9065fe58862f22a8e2a257f0465e17c1f55dd7fdbf5bf2ddd6652eada2"}
Jul 21 13:10:31.715324 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:10:31.715151    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-1f53c0d5-openshift-default-cc8f47c49-m84lh" event={"ID":"a781e1f8-c90e-4825-bbf0-3b484128736b","Type":"ContainerDied","Data":"f2e3ccddb6570f8bab072d81a9595f293b9b1a5bab0b7ba59be991311a1b7db4"}
Jul 21 13:10:31.715324 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:10:31.715171    2556 scope.go:117] "RemoveContainer" containerID="ef934c9065fe58862f22a8e2a257f0465e17c1f55dd7fdbf5bf2ddd6652eada2"
Jul 21 13:10:31.725474 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:10:31.725458    2556 scope.go:117] "RemoveContainer" containerID="d39f9a91b354dca4f5f6123630d277c3ac646e33045763f50752fccf3a0a82a5"
Jul 21 13:10:31.733961 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:10:31.733945    2556 scope.go:117] "RemoveContainer" containerID="ef934c9065fe58862f22a8e2a257f0465e17c1f55dd7fdbf5bf2ddd6652eada2"
Jul 21 13:10:31.734245 ip-10-0-130-2 kubenswrapper[2556]: E0721 13:10:31.734228    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ef934c9065fe58862f22a8e2a257f0465e17c1f55dd7fdbf5bf2ddd6652eada2\": container with ID starting with ef934c9065fe58862f22a8e2a257f0465e17c1f55dd7fdbf5bf2ddd6652eada2 not found: ID does not exist" containerID="ef934c9065fe58862f22a8e2a257f0465e17c1f55dd7fdbf5bf2ddd6652eada2"
Jul 21 13:10:31.734327 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:10:31.734257    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ef934c9065fe58862f22a8e2a257f0465e17c1f55dd7fdbf5bf2ddd6652eada2"} err="failed to get container status \"ef934c9065fe58862f22a8e2a257f0465e17c1f55dd7fdbf5bf2ddd6652eada2\": rpc error: code = NotFound desc = could not find container \"ef934c9065fe58862f22a8e2a257f0465e17c1f55dd7fdbf5bf2ddd6652eada2\": container with ID starting with ef934c9065fe58862f22a8e2a257f0465e17c1f55dd7fdbf5bf2ddd6652eada2 not found: ID does not exist"
Jul 21 13:10:31.734327 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:10:31.734280    2556 scope.go:117] "RemoveContainer" containerID="d39f9a91b354dca4f5f6123630d277c3ac646e33045763f50752fccf3a0a82a5"
Jul 21 13:10:31.734475 ip-10-0-130-2 kubenswrapper[2556]: E0721 13:10:31.734456    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d39f9a91b354dca4f5f6123630d277c3ac646e33045763f50752fccf3a0a82a5\": container with ID starting with d39f9a91b354dca4f5f6123630d277c3ac646e33045763f50752fccf3a0a82a5 not found: ID does not exist" containerID="d39f9a91b354dca4f5f6123630d277c3ac646e33045763f50752fccf3a0a82a5"
Jul 21 13:10:31.734518 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:10:31.734482    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d39f9a91b354dca4f5f6123630d277c3ac646e33045763f50752fccf3a0a82a5"} err="failed to get container status \"d39f9a91b354dca4f5f6123630d277c3ac646e33045763f50752fccf3a0a82a5\": rpc error: code = NotFound desc = could not find container \"d39f9a91b354dca4f5f6123630d277c3ac646e33045763f50752fccf3a0a82a5\": container with ID starting with d39f9a91b354dca4f5f6123630d277c3ac646e33045763f50752fccf3a0a82a5 not found: ID does not exist"
Jul 21 13:10:31.740100 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:10:31.740078    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-1f53c0d5-openshift-default-cc8f47c49-m84lh"]
Jul 21 13:10:31.747620 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:10:31.747598    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-1f53c0d5-openshift-default-cc8f47c49-m84lh"]
Jul 21 13:10:33.197035 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:10:33.195563    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a781e1f8-c90e-4825-bbf0-3b484128736b" path="/var/lib/kubelet/pods/a781e1f8-c90e-4825-bbf0-3b484128736b/volumes"
Jul 21 13:10:53.677911 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:10:53.677878    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-c06656ac-qnr2c"]
Jul 21 13:10:53.678305 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:10:53.678292    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a781e1f8-c90e-4825-bbf0-3b484128736b" containerName="istio-proxy"
Jul 21 13:10:53.678349 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:10:53.678306    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="a781e1f8-c90e-4825-bbf0-3b484128736b" containerName="istio-proxy"
Jul 21 13:10:53.678349 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:10:53.678333    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a97223f7-9972-4345-b6cc-c441c90a39d1" containerName="revoke-keys"
Jul 21 13:10:53.678349 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:10:53.678339    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="a97223f7-9972-4345-b6cc-c441c90a39d1" containerName="revoke-keys"
Jul 21 13:10:53.678446 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:10:53.678411    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="a781e1f8-c90e-4825-bbf0-3b484128736b" containerName="istio-proxy"
Jul 21 13:10:53.678446 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:10:53.678421    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="a781e1f8-c90e-4825-bbf0-3b484128736b" containerName="istio-proxy"
Jul 21 13:10:53.678446 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:10:53.678428    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="a97223f7-9972-4345-b6cc-c441c90a39d1" containerName="revoke-keys"
Jul 21 13:10:53.681577 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:10:53.681562    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-c06656ac-qnr2c"
Jul 21 13:10:53.684005 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:10:53.683985    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5nsrr\""
Jul 21 13:10:53.692390 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:10:53.692369    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-c06656ac-qnr2c"]
Jul 21 13:10:53.711083 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:10:53.711056    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ff6be943-e4bf-4104-9fe9-8921e8a773db-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-c06656ac-qnr2c\" (UID: \"ff6be943-e4bf-4104-9fe9-8921e8a773db\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-c06656ac-qnr2c"
Jul 21 13:10:53.811603 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:10:53.811578    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ff6be943-e4bf-4104-9fe9-8921e8a773db-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-c06656ac-qnr2c\" (UID: \"ff6be943-e4bf-4104-9fe9-8921e8a773db\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-c06656ac-qnr2c"
Jul 21 13:10:53.812178 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:10:53.812159    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ff6be943-e4bf-4104-9fe9-8921e8a773db-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-c06656ac-qnr2c\" (UID: \"ff6be943-e4bf-4104-9fe9-8921e8a773db\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-c06656ac-qnr2c"
Jul 21 13:10:53.992278 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:10:53.992248    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-c06656ac-qnr2c"
Jul 21 13:10:54.134427 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:10:54.134403    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-c06656ac-qnr2c"]
Jul 21 13:10:54.136508 ip-10-0-130-2 kubenswrapper[2556]: W0721 13:10:54.136474    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff6be943_e4bf_4104_9fe9_8921e8a773db.slice/crio-f68cb1b404b6ea0754f6e96fab030d8cc18f83c1cdb4c20369a48d2666f3e0c9 WatchSource:0}: Error finding container f68cb1b404b6ea0754f6e96fab030d8cc18f83c1cdb4c20369a48d2666f3e0c9: Status 404 returned error can't find the container with id f68cb1b404b6ea0754f6e96fab030d8cc18f83c1cdb4c20369a48d2666f3e0c9
Jul 21 13:10:54.845327 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:10:54.845296    2556 generic.go:356] "Generic (PLEG): container finished" podID="ff6be943-e4bf-4104-9fe9-8921e8a773db" containerID="3d21f31c3275b33db336fdfeaae59e9f56fa99d1ea91bd843ee715708283377a" exitCode=0
Jul 21 13:10:54.845653 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:10:54.845376    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-c06656ac-qnr2c" event={"ID":"ff6be943-e4bf-4104-9fe9-8921e8a773db","Type":"ContainerDied","Data":"3d21f31c3275b33db336fdfeaae59e9f56fa99d1ea91bd843ee715708283377a"}
Jul 21 13:10:54.845653 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:10:54.845441    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-c06656ac-qnr2c" event={"ID":"ff6be943-e4bf-4104-9fe9-8921e8a773db","Type":"ContainerStarted","Data":"f68cb1b404b6ea0754f6e96fab030d8cc18f83c1cdb4c20369a48d2666f3e0c9"}
Jul 21 13:10:55.989378 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:10:55.989357    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-c06656ac-qnr2c"
Jul 21 13:10:56.029512 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:10:56.029490    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ff6be943-e4bf-4104-9fe9-8921e8a773db-maas-api-service-ca\") pod \"ff6be943-e4bf-4104-9fe9-8921e8a773db\" (UID: \"ff6be943-e4bf-4104-9fe9-8921e8a773db\") "
Jul 21 13:10:56.029831 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:10:56.029800    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ff6be943-e4bf-4104-9fe9-8921e8a773db-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "ff6be943-e4bf-4104-9fe9-8921e8a773db" (UID: "ff6be943-e4bf-4104-9fe9-8921e8a773db"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 13:10:56.130374 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:10:56.130310    2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ff6be943-e4bf-4104-9fe9-8921e8a773db-maas-api-service-ca\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Jul 21 13:10:56.856714 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:10:56.856691    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-c06656ac-qnr2c"
Jul 21 13:10:56.856855 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:10:56.856734    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-c06656ac-qnr2c" event={"ID":"ff6be943-e4bf-4104-9fe9-8921e8a773db","Type":"ContainerDied","Data":"f68cb1b404b6ea0754f6e96fab030d8cc18f83c1cdb4c20369a48d2666f3e0c9"}
Jul 21 13:10:56.856855 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:10:56.856808    2556 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f68cb1b404b6ea0754f6e96fab030d8cc18f83c1cdb4c20369a48d2666f3e0c9"
Jul 21 13:11:08.788542 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:11:08.788459    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-c06656ac-qnr2c"]
Jul 21 13:11:08.792736 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:11:08.792707    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-c06656ac-qnr2c"]
Jul 21 13:11:09.199845 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:11:09.199757    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ff6be943-e4bf-4104-9fe9-8921e8a773db" path="/var/lib/kubelet/pods/ff6be943-e4bf-4104-9fe9-8921e8a773db/volumes"
Jul 21 13:13:12.652275 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:13:12.652247    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4de9b363-8gxg7"]
Jul 21 13:13:12.652702 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:13:12.652666    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ff6be943-e4bf-4104-9fe9-8921e8a773db" containerName="revoke-keys"
Jul 21 13:13:12.652702 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:13:12.652677    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="ff6be943-e4bf-4104-9fe9-8921e8a773db" containerName="revoke-keys"
Jul 21 13:13:12.652702 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:13:12.652687    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a781e1f8-c90e-4825-bbf0-3b484128736b" containerName="istio-proxy"
Jul 21 13:13:12.652702 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:13:12.652693    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="a781e1f8-c90e-4825-bbf0-3b484128736b" containerName="istio-proxy"
Jul 21 13:13:12.652865 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:13:12.652749    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="ff6be943-e4bf-4104-9fe9-8921e8a773db" containerName="revoke-keys"
Jul 21 13:13:12.655935 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:13:12.655911    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4de9b363-8gxg7"
Jul 21 13:13:12.657916 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:13:12.657898    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5nsrr\""
Jul 21 13:13:12.664158 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:13:12.664134    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4de9b363-8gxg7"]
Jul 21 13:13:12.710115 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:13:12.710090    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e0442542-5c12-48be-b383-fa1a0895a395-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-4de9b363-8gxg7\" (UID: \"e0442542-5c12-48be-b383-fa1a0895a395\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4de9b363-8gxg7"
Jul 21 13:13:12.810811 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:13:12.810789    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e0442542-5c12-48be-b383-fa1a0895a395-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-4de9b363-8gxg7\" (UID: \"e0442542-5c12-48be-b383-fa1a0895a395\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4de9b363-8gxg7"
Jul 21 13:13:12.811380 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:13:12.811361    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e0442542-5c12-48be-b383-fa1a0895a395-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-4de9b363-8gxg7\" (UID: \"e0442542-5c12-48be-b383-fa1a0895a395\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4de9b363-8gxg7"
Jul 21 13:13:12.968664 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:13:12.968608    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4de9b363-8gxg7"
Jul 21 13:13:13.316303 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:13:13.316275    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4de9b363-8gxg7"]
Jul 21 13:13:13.317477 ip-10-0-130-2 kubenswrapper[2556]: W0721 13:13:13.317452    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0442542_5c12_48be_b383_fa1a0895a395.slice/crio-e1dbca47c99b0274a7a4802ae77d36fd848cfb6803208578446ead867cac66e5 WatchSource:0}: Error finding container e1dbca47c99b0274a7a4802ae77d36fd848cfb6803208578446ead867cac66e5: Status 404 returned error can't find the container with id e1dbca47c99b0274a7a4802ae77d36fd848cfb6803208578446ead867cac66e5
Jul 21 13:13:13.617661 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:13:13.617636    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4de9b363-8gxg7" event={"ID":"e0442542-5c12-48be-b383-fa1a0895a395","Type":"ContainerStarted","Data":"4eec062d8115e0bd372a0e051f2d5e4e2d01a7a72ba8769a14b7008fa075d55e"}
Jul 21 13:13:13.617769 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:13:13.617686    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4de9b363-8gxg7" event={"ID":"e0442542-5c12-48be-b383-fa1a0895a395","Type":"ContainerStarted","Data":"e1dbca47c99b0274a7a4802ae77d36fd848cfb6803208578446ead867cac66e5"}
Jul 21 13:13:13.635093 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:13:13.635051    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4de9b363-8gxg7" podStartSLOduration=1.635035196 podStartE2EDuration="1.635035196s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 13:13:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:13:13.634343807 +0000 UTC m=+2006.938348897" watchObservedRunningTime="2026-07-21 13:13:13.635035196 +0000 UTC m=+2006.939040283"
Jul 21 13:13:14.624775 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:13:14.624741    2556 generic.go:356] "Generic (PLEG): container finished" podID="e0442542-5c12-48be-b383-fa1a0895a395" containerID="4eec062d8115e0bd372a0e051f2d5e4e2d01a7a72ba8769a14b7008fa075d55e" exitCode=0
Jul 21 13:13:14.625154 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:13:14.624828    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4de9b363-8gxg7" event={"ID":"e0442542-5c12-48be-b383-fa1a0895a395","Type":"ContainerDied","Data":"4eec062d8115e0bd372a0e051f2d5e4e2d01a7a72ba8769a14b7008fa075d55e"}
Jul 21 13:13:15.770481 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:13:15.770452    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4de9b363-8gxg7"
Jul 21 13:13:15.834965 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:13:15.834938    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e0442542-5c12-48be-b383-fa1a0895a395-maas-api-service-ca\") pod \"e0442542-5c12-48be-b383-fa1a0895a395\" (UID: \"e0442542-5c12-48be-b383-fa1a0895a395\") "
Jul 21 13:13:15.835285 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:13:15.835264    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e0442542-5c12-48be-b383-fa1a0895a395-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e0442542-5c12-48be-b383-fa1a0895a395" (UID: "e0442542-5c12-48be-b383-fa1a0895a395"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 13:13:15.936521 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:13:15.936464    2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e0442542-5c12-48be-b383-fa1a0895a395-maas-api-service-ca\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Jul 21 13:13:16.637488 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:13:16.637457    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4de9b363-8gxg7"
Jul 21 13:13:16.637637 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:13:16.637483    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4de9b363-8gxg7" event={"ID":"e0442542-5c12-48be-b383-fa1a0895a395","Type":"ContainerDied","Data":"e1dbca47c99b0274a7a4802ae77d36fd848cfb6803208578446ead867cac66e5"}
Jul 21 13:13:16.637637 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:13:16.637534    2556 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e1dbca47c99b0274a7a4802ae77d36fd848cfb6803208578446ead867cac66e5"
Jul 21 13:13:27.767763 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:13:27.767721    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4de9b363-8gxg7"]
Jul 21 13:13:27.768341 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:13:27.767782    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4de9b363-8gxg7"]
Jul 21 13:13:29.194547 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:13:29.194507    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e0442542-5c12-48be-b383-fa1a0895a395" path="/var/lib/kubelet/pods/e0442542-5c12-48be-b383-fa1a0895a395/volumes"
Jul 21 13:14:20.048903 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:14:20.048821    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-4f0dda7a/e2e-lifecycle-model-4f0dda7a-kserve-6445c54499-9tl6c"]
Jul 21 13:14:20.049373 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:14:20.049283    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e0442542-5c12-48be-b383-fa1a0895a395" containerName="revoke-keys"
Jul 21 13:14:20.049373 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:14:20.049296    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="e0442542-5c12-48be-b383-fa1a0895a395" containerName="revoke-keys"
Jul 21 13:14:20.049449 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:14:20.049381    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="e0442542-5c12-48be-b383-fa1a0895a395" containerName="revoke-keys"
Jul 21 13:14:20.052475 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:14:20.052456    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-4f0dda7a/e2e-lifecycle-model-4f0dda7a-kserve-6445c54499-9tl6c"
Jul 21 13:14:20.054986 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:14:20.054960    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-4f0dda7a\"/\"kube-root-ca.crt\""
Jul 21 13:14:20.055470 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:14:20.055441    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-4f0dda7a\"/\"openshift-service-ca.crt\""
Jul 21 13:14:20.055571 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:14:20.055520    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-4f0dda7a\"/\"default-dockercfg-rhdd8\""
Jul 21 13:14:20.055571 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:14:20.055521    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-4f0dda7a\"/\"e2e-lifecycle-model-4f0dda7a-kserve-self-signed-certs\""
Jul 21 13:14:20.063880 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:14:20.063858    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-4f0dda7a/e2e-lifecycle-model-4f0dda7a-kserve-6445c54499-9tl6c"]
Jul 21 13:14:20.218633 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:14:20.218607    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5plz8\" (UniqueName: \"kubernetes.io/projected/45dc2582-a116-4520-9dac-c5d60e2d28b4-kube-api-access-5plz8\") pod \"e2e-lifecycle-model-4f0dda7a-kserve-6445c54499-9tl6c\" (UID: \"45dc2582-a116-4520-9dac-c5d60e2d28b4\") " pod="ai-tenant-e2e-mt-4f0dda7a/e2e-lifecycle-model-4f0dda7a-kserve-6445c54499-9tl6c"
Jul 21 13:14:20.218772 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:14:20.218644    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/45dc2582-a116-4520-9dac-c5d60e2d28b4-dshm\") pod \"e2e-lifecycle-model-4f0dda7a-kserve-6445c54499-9tl6c\" (UID: \"45dc2582-a116-4520-9dac-c5d60e2d28b4\") " pod="ai-tenant-e2e-mt-4f0dda7a/e2e-lifecycle-model-4f0dda7a-kserve-6445c54499-9tl6c"
Jul 21 13:14:20.218772 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:14:20.218662    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/45dc2582-a116-4520-9dac-c5d60e2d28b4-tls-certs\") pod \"e2e-lifecycle-model-4f0dda7a-kserve-6445c54499-9tl6c\" (UID: \"45dc2582-a116-4520-9dac-c5d60e2d28b4\") " pod="ai-tenant-e2e-mt-4f0dda7a/e2e-lifecycle-model-4f0dda7a-kserve-6445c54499-9tl6c"
Jul 21 13:14:20.218772 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:14:20.218683    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/45dc2582-a116-4520-9dac-c5d60e2d28b4-home\") pod \"e2e-lifecycle-model-4f0dda7a-kserve-6445c54499-9tl6c\" (UID: \"45dc2582-a116-4520-9dac-c5d60e2d28b4\") " pod="ai-tenant-e2e-mt-4f0dda7a/e2e-lifecycle-model-4f0dda7a-kserve-6445c54499-9tl6c"
Jul 21 13:14:20.218772 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:14:20.218699    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/45dc2582-a116-4520-9dac-c5d60e2d28b4-tmp-dir\") pod \"e2e-lifecycle-model-4f0dda7a-kserve-6445c54499-9tl6c\" (UID: \"45dc2582-a116-4520-9dac-c5d60e2d28b4\") " pod="ai-tenant-e2e-mt-4f0dda7a/e2e-lifecycle-model-4f0dda7a-kserve-6445c54499-9tl6c"
Jul 21 13:14:20.218772 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:14:20.218741    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/45dc2582-a116-4520-9dac-c5d60e2d28b4-model-cache\") pod \"e2e-lifecycle-model-4f0dda7a-kserve-6445c54499-9tl6c\" (UID: \"45dc2582-a116-4520-9dac-c5d60e2d28b4\") " pod="ai-tenant-e2e-mt-4f0dda7a/e2e-lifecycle-model-4f0dda7a-kserve-6445c54499-9tl6c"
Jul 21 13:14:20.320216 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:14:20.320131    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/45dc2582-a116-4520-9dac-c5d60e2d28b4-model-cache\") pod \"e2e-lifecycle-model-4f0dda7a-kserve-6445c54499-9tl6c\" (UID: \"45dc2582-a116-4520-9dac-c5d60e2d28b4\") " pod="ai-tenant-e2e-mt-4f0dda7a/e2e-lifecycle-model-4f0dda7a-kserve-6445c54499-9tl6c"
Jul 21 13:14:20.320329 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:14:20.320221    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5plz8\" (UniqueName: \"kubernetes.io/projected/45dc2582-a116-4520-9dac-c5d60e2d28b4-kube-api-access-5plz8\") pod \"e2e-lifecycle-model-4f0dda7a-kserve-6445c54499-9tl6c\" (UID: \"45dc2582-a116-4520-9dac-c5d60e2d28b4\") " pod="ai-tenant-e2e-mt-4f0dda7a/e2e-lifecycle-model-4f0dda7a-kserve-6445c54499-9tl6c"
Jul 21 13:14:20.320329 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:14:20.320274    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/45dc2582-a116-4520-9dac-c5d60e2d28b4-dshm\") pod \"e2e-lifecycle-model-4f0dda7a-kserve-6445c54499-9tl6c\" (UID: \"45dc2582-a116-4520-9dac-c5d60e2d28b4\") " pod="ai-tenant-e2e-mt-4f0dda7a/e2e-lifecycle-model-4f0dda7a-kserve-6445c54499-9tl6c"
Jul 21 13:14:20.320329 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:14:20.320291    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/45dc2582-a116-4520-9dac-c5d60e2d28b4-tls-certs\") pod \"e2e-lifecycle-model-4f0dda7a-kserve-6445c54499-9tl6c\" (UID: \"45dc2582-a116-4520-9dac-c5d60e2d28b4\") " pod="ai-tenant-e2e-mt-4f0dda7a/e2e-lifecycle-model-4f0dda7a-kserve-6445c54499-9tl6c"
Jul 21 13:14:20.320329 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:14:20.320315    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/45dc2582-a116-4520-9dac-c5d60e2d28b4-home\") pod \"e2e-lifecycle-model-4f0dda7a-kserve-6445c54499-9tl6c\" (UID: \"45dc2582-a116-4520-9dac-c5d60e2d28b4\") " pod="ai-tenant-e2e-mt-4f0dda7a/e2e-lifecycle-model-4f0dda7a-kserve-6445c54499-9tl6c"
Jul 21 13:14:20.320552 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:14:20.320331    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/45dc2582-a116-4520-9dac-c5d60e2d28b4-tmp-dir\") pod \"e2e-lifecycle-model-4f0dda7a-kserve-6445c54499-9tl6c\" (UID: \"45dc2582-a116-4520-9dac-c5d60e2d28b4\") " pod="ai-tenant-e2e-mt-4f0dda7a/e2e-lifecycle-model-4f0dda7a-kserve-6445c54499-9tl6c"
Jul 21 13:14:20.320611 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:14:20.320561    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/45dc2582-a116-4520-9dac-c5d60e2d28b4-model-cache\") pod \"e2e-lifecycle-model-4f0dda7a-kserve-6445c54499-9tl6c\" (UID: \"45dc2582-a116-4520-9dac-c5d60e2d28b4\") " pod="ai-tenant-e2e-mt-4f0dda7a/e2e-lifecycle-model-4f0dda7a-kserve-6445c54499-9tl6c"
Jul 21 13:14:20.320708 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:14:20.320688    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/45dc2582-a116-4520-9dac-c5d60e2d28b4-home\") pod \"e2e-lifecycle-model-4f0dda7a-kserve-6445c54499-9tl6c\" (UID: \"45dc2582-a116-4520-9dac-c5d60e2d28b4\") " pod="ai-tenant-e2e-mt-4f0dda7a/e2e-lifecycle-model-4f0dda7a-kserve-6445c54499-9tl6c"
Jul 21 13:14:20.320801 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:14:20.320783    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/45dc2582-a116-4520-9dac-c5d60e2d28b4-tmp-dir\") pod \"e2e-lifecycle-model-4f0dda7a-kserve-6445c54499-9tl6c\" (UID: \"45dc2582-a116-4520-9dac-c5d60e2d28b4\") " pod="ai-tenant-e2e-mt-4f0dda7a/e2e-lifecycle-model-4f0dda7a-kserve-6445c54499-9tl6c"
Jul 21 13:14:20.322691 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:14:20.322668    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/45dc2582-a116-4520-9dac-c5d60e2d28b4-dshm\") pod \"e2e-lifecycle-model-4f0dda7a-kserve-6445c54499-9tl6c\" (UID: \"45dc2582-a116-4520-9dac-c5d60e2d28b4\") " pod="ai-tenant-e2e-mt-4f0dda7a/e2e-lifecycle-model-4f0dda7a-kserve-6445c54499-9tl6c"
Jul 21 13:14:20.323003 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:14:20.322987    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/45dc2582-a116-4520-9dac-c5d60e2d28b4-tls-certs\") pod \"e2e-lifecycle-model-4f0dda7a-kserve-6445c54499-9tl6c\" (UID: \"45dc2582-a116-4520-9dac-c5d60e2d28b4\") " pod="ai-tenant-e2e-mt-4f0dda7a/e2e-lifecycle-model-4f0dda7a-kserve-6445c54499-9tl6c"
Jul 21 13:14:20.329870 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:14:20.329851    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5plz8\" (UniqueName: \"kubernetes.io/projected/45dc2582-a116-4520-9dac-c5d60e2d28b4-kube-api-access-5plz8\") pod \"e2e-lifecycle-model-4f0dda7a-kserve-6445c54499-9tl6c\" (UID: \"45dc2582-a116-4520-9dac-c5d60e2d28b4\") " pod="ai-tenant-e2e-mt-4f0dda7a/e2e-lifecycle-model-4f0dda7a-kserve-6445c54499-9tl6c"
Jul 21 13:14:20.362078 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:14:20.362053    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-4f0dda7a/e2e-lifecycle-model-4f0dda7a-kserve-6445c54499-9tl6c"
Jul 21 13:14:20.504303 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:14:20.504276    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-4f0dda7a/e2e-lifecycle-model-4f0dda7a-kserve-6445c54499-9tl6c"]
Jul 21 13:14:20.506820 ip-10-0-130-2 kubenswrapper[2556]: W0721 13:14:20.506791    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod45dc2582_a116_4520_9dac_c5d60e2d28b4.slice/crio-5608ebd86d674431665185fa01d9493a437dc355c99523289959dd65e08bee74 WatchSource:0}: Error finding container 5608ebd86d674431665185fa01d9493a437dc355c99523289959dd65e08bee74: Status 404 returned error can't find the container with id 5608ebd86d674431665185fa01d9493a437dc355c99523289959dd65e08bee74
Jul 21 13:14:21.007984 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:14:21.007946    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-4f0dda7a/e2e-lifecycle-model-4f0dda7a-kserve-6445c54499-9tl6c" event={"ID":"45dc2582-a116-4520-9dac-c5d60e2d28b4","Type":"ContainerStarted","Data":"231bee3bf39893037b77de23f285326703125f163eb3f3088a1059a309e4891e"}
Jul 21 13:14:21.008169 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:14:21.007995    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-4f0dda7a/e2e-lifecycle-model-4f0dda7a-kserve-6445c54499-9tl6c" event={"ID":"45dc2582-a116-4520-9dac-c5d60e2d28b4","Type":"ContainerStarted","Data":"5608ebd86d674431665185fa01d9493a437dc355c99523289959dd65e08bee74"}
Jul 21 13:14:21.034029 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:14:21.033955    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-4f0dda7a/e2e-lifecycle-model-4f0dda7a-kserve-6445c54499-9tl6c" podStartSLOduration=1.03393738 podStartE2EDuration="1.03393738s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 13:14:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:14:21.032592347 +0000 UTC m=+2074.336597437" watchObservedRunningTime="2026-07-21 13:14:21.03393738 +0000 UTC m=+2074.337942472"
Jul 21 13:14:30.362531 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:14:30.362495    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-4f0dda7a/e2e-lifecycle-model-4f0dda7a-kserve-6445c54499-9tl6c"
Jul 21 13:14:30.362948 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:14:30.362547    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-4f0dda7a/e2e-lifecycle-model-4f0dda7a-kserve-6445c54499-9tl6c"
Jul 21 13:14:30.375266 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:14:30.375241    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-4f0dda7a/e2e-lifecycle-model-4f0dda7a-kserve-6445c54499-9tl6c"
Jul 21 13:14:31.080133 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:14:31.080104    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-4f0dda7a/e2e-lifecycle-model-4f0dda7a-kserve-6445c54499-9tl6c"
Jul 21 13:14:41.619455 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:14:41.619414    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4f0dda7a-b2s8f"]
Jul 21 13:14:41.623340 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:14:41.623320    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4f0dda7a-b2s8f"
Jul 21 13:14:41.626780 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:14:41.626757    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5nsrr\""
Jul 21 13:14:41.636669 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:14:41.636640    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4f0dda7a-b2s8f"]
Jul 21 13:14:41.714877 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:14:41.714834    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6d1dcc9d-b95a-4412-b460-a17ea6cd0fb0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-4f0dda7a-b2s8f\" (UID: \"6d1dcc9d-b95a-4412-b460-a17ea6cd0fb0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4f0dda7a-b2s8f"
Jul 21 13:14:41.815964 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:14:41.815923    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6d1dcc9d-b95a-4412-b460-a17ea6cd0fb0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-4f0dda7a-b2s8f\" (UID: \"6d1dcc9d-b95a-4412-b460-a17ea6cd0fb0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4f0dda7a-b2s8f"
Jul 21 13:14:41.816570 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:14:41.816543    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6d1dcc9d-b95a-4412-b460-a17ea6cd0fb0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-4f0dda7a-b2s8f\" (UID: \"6d1dcc9d-b95a-4412-b460-a17ea6cd0fb0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4f0dda7a-b2s8f"
Jul 21 13:14:41.938407 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:14:41.938290    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4f0dda7a-b2s8f"
Jul 21 13:14:42.081852 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:14:42.081829    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4f0dda7a-b2s8f"]
Jul 21 13:14:42.083849 ip-10-0-130-2 kubenswrapper[2556]: W0721 13:14:42.083815    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d1dcc9d_b95a_4412_b460_a17ea6cd0fb0.slice/crio-7b6b43199f54d46feb9ae4d5e433ccb098a017b09773152fdf6ac7312646064f WatchSource:0}: Error finding container 7b6b43199f54d46feb9ae4d5e433ccb098a017b09773152fdf6ac7312646064f: Status 404 returned error can't find the container with id 7b6b43199f54d46feb9ae4d5e433ccb098a017b09773152fdf6ac7312646064f
Jul 21 13:14:42.128688 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:14:42.128653    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4f0dda7a-b2s8f" event={"ID":"6d1dcc9d-b95a-4412-b460-a17ea6cd0fb0","Type":"ContainerStarted","Data":"7b6b43199f54d46feb9ae4d5e433ccb098a017b09773152fdf6ac7312646064f"}
Jul 21 13:14:43.135740 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:14:43.135658    2556 generic.go:356] "Generic (PLEG): container finished" podID="6d1dcc9d-b95a-4412-b460-a17ea6cd0fb0" containerID="6be8aff0640f245550a9603a9e6fba9a8237e3878dc12824e3d3b4d3fa92ddc9" exitCode=0
Jul 21 13:14:43.136116 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:14:43.135739    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4f0dda7a-b2s8f" event={"ID":"6d1dcc9d-b95a-4412-b460-a17ea6cd0fb0","Type":"ContainerDied","Data":"6be8aff0640f245550a9603a9e6fba9a8237e3878dc12824e3d3b4d3fa92ddc9"}
Jul 21 13:14:44.275290 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:14:44.275266    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4f0dda7a-b2s8f"
Jul 21 13:14:44.341396 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:14:44.341354    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6d1dcc9d-b95a-4412-b460-a17ea6cd0fb0-maas-api-service-ca\") pod \"6d1dcc9d-b95a-4412-b460-a17ea6cd0fb0\" (UID: \"6d1dcc9d-b95a-4412-b460-a17ea6cd0fb0\") "
Jul 21 13:14:44.341706 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:14:44.341686    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6d1dcc9d-b95a-4412-b460-a17ea6cd0fb0-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "6d1dcc9d-b95a-4412-b460-a17ea6cd0fb0" (UID: "6d1dcc9d-b95a-4412-b460-a17ea6cd0fb0"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 13:14:44.442775 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:14:44.442709    2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6d1dcc9d-b95a-4412-b460-a17ea6cd0fb0-maas-api-service-ca\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Jul 21 13:14:45.149486 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:14:45.149461    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4f0dda7a-b2s8f"
Jul 21 13:14:45.149649 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:14:45.149499    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4f0dda7a-b2s8f" event={"ID":"6d1dcc9d-b95a-4412-b460-a17ea6cd0fb0","Type":"ContainerDied","Data":"7b6b43199f54d46feb9ae4d5e433ccb098a017b09773152fdf6ac7312646064f"}
Jul 21 13:14:45.149649 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:14:45.149551    2556 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7b6b43199f54d46feb9ae4d5e433ccb098a017b09773152fdf6ac7312646064f"
Jul 21 13:14:47.383074 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:14:47.383042    2556 scope.go:117] "RemoveContainer" containerID="d990b84c107b0c46059d7b9bb2c990abcbdd961a3f50ebfcb45d2dfe9980ca55"
Jul 21 13:15:01.747385 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:15:01.747352    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4f0dda7a-b2s8f"]
Jul 21 13:15:01.750217 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:15:01.750198    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4f0dda7a-b2s8f"]
Jul 21 13:15:03.196106 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:15:03.196070    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6d1dcc9d-b95a-4412-b460-a17ea6cd0fb0" path="/var/lib/kubelet/pods/6d1dcc9d-b95a-4412-b460-a17ea6cd0fb0/volumes"
Jul 21 13:15:09.016296 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:15:09.016254    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-4f0dda7a/e2e-lifecycle-model-4f0dda7a-kserve-6445c54499-9tl6c"]
Jul 21 13:15:24.038877 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:15:24.038827    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-4f0dda7a/e2e-lifecycle-model-4f0dda7a-kserve-6445c54499-9tl6c" podUID="45dc2582-a116-4520-9dac-c5d60e2d28b4" containerName="main" containerID="cri-o://231bee3bf39893037b77de23f285326703125f163eb3f3088a1059a309e4891e" gracePeriod=45
Jul 21 13:15:24.299750 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:15:24.299691    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-4f0dda7a/e2e-lifecycle-model-4f0dda7a-kserve-6445c54499-9tl6c"
Jul 21 13:15:24.375520 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:15:24.375494    2556 generic.go:356] "Generic (PLEG): container finished" podID="45dc2582-a116-4520-9dac-c5d60e2d28b4" containerID="231bee3bf39893037b77de23f285326703125f163eb3f3088a1059a309e4891e" exitCode=0
Jul 21 13:15:24.375678 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:15:24.375535    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-4f0dda7a/e2e-lifecycle-model-4f0dda7a-kserve-6445c54499-9tl6c"
Jul 21 13:15:24.375678 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:15:24.375540    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-4f0dda7a/e2e-lifecycle-model-4f0dda7a-kserve-6445c54499-9tl6c" event={"ID":"45dc2582-a116-4520-9dac-c5d60e2d28b4","Type":"ContainerDied","Data":"231bee3bf39893037b77de23f285326703125f163eb3f3088a1059a309e4891e"}
Jul 21 13:15:24.375678 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:15:24.375669    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-4f0dda7a/e2e-lifecycle-model-4f0dda7a-kserve-6445c54499-9tl6c" event={"ID":"45dc2582-a116-4520-9dac-c5d60e2d28b4","Type":"ContainerDied","Data":"5608ebd86d674431665185fa01d9493a437dc355c99523289959dd65e08bee74"}
Jul 21 13:15:24.375852 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:15:24.375700    2556 scope.go:117] "RemoveContainer" containerID="231bee3bf39893037b77de23f285326703125f163eb3f3088a1059a309e4891e"
Jul 21 13:15:24.380632 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:15:24.380613    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/45dc2582-a116-4520-9dac-c5d60e2d28b4-model-cache\") pod \"45dc2582-a116-4520-9dac-c5d60e2d28b4\" (UID: \"45dc2582-a116-4520-9dac-c5d60e2d28b4\") "
Jul 21 13:15:24.380738 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:15:24.380666    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/45dc2582-a116-4520-9dac-c5d60e2d28b4-home\") pod \"45dc2582-a116-4520-9dac-c5d60e2d28b4\" (UID: \"45dc2582-a116-4520-9dac-c5d60e2d28b4\") "
Jul 21 13:15:24.380738 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:15:24.380700    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/45dc2582-a116-4520-9dac-c5d60e2d28b4-tls-certs\") pod \"45dc2582-a116-4520-9dac-c5d60e2d28b4\" (UID: \"45dc2582-a116-4520-9dac-c5d60e2d28b4\") "
Jul 21 13:15:24.380856 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:15:24.380765    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5plz8\" (UniqueName: \"kubernetes.io/projected/45dc2582-a116-4520-9dac-c5d60e2d28b4-kube-api-access-5plz8\") pod \"45dc2582-a116-4520-9dac-c5d60e2d28b4\" (UID: \"45dc2582-a116-4520-9dac-c5d60e2d28b4\") "
Jul 21 13:15:24.380856 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:15:24.380806    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/45dc2582-a116-4520-9dac-c5d60e2d28b4-dshm\") pod \"45dc2582-a116-4520-9dac-c5d60e2d28b4\" (UID: \"45dc2582-a116-4520-9dac-c5d60e2d28b4\") "
Jul 21 13:15:24.380856 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:15:24.380830    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/45dc2582-a116-4520-9dac-c5d60e2d28b4-tmp-dir\") pod \"45dc2582-a116-4520-9dac-c5d60e2d28b4\" (UID: \"45dc2582-a116-4520-9dac-c5d60e2d28b4\") "
Jul 21 13:15:24.381034 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:15:24.380888    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/45dc2582-a116-4520-9dac-c5d60e2d28b4-model-cache" (OuterVolumeSpecName: "model-cache") pod "45dc2582-a116-4520-9dac-c5d60e2d28b4" (UID: "45dc2582-a116-4520-9dac-c5d60e2d28b4"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 13:15:24.381034 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:15:24.380938    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/45dc2582-a116-4520-9dac-c5d60e2d28b4-home" (OuterVolumeSpecName: "home") pod "45dc2582-a116-4520-9dac-c5d60e2d28b4" (UID: "45dc2582-a116-4520-9dac-c5d60e2d28b4"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 13:15:24.381184 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:15:24.381161    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/45dc2582-a116-4520-9dac-c5d60e2d28b4-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "45dc2582-a116-4520-9dac-c5d60e2d28b4" (UID: "45dc2582-a116-4520-9dac-c5d60e2d28b4"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 13:15:24.381245 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:15:24.381229    2556 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/45dc2582-a116-4520-9dac-c5d60e2d28b4-home\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Jul 21 13:15:24.381295 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:15:24.381253    2556 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/45dc2582-a116-4520-9dac-c5d60e2d28b4-model-cache\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Jul 21 13:15:24.382893 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:15:24.382872    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/45dc2582-a116-4520-9dac-c5d60e2d28b4-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "45dc2582-a116-4520-9dac-c5d60e2d28b4" (UID: "45dc2582-a116-4520-9dac-c5d60e2d28b4"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 13:15:24.383040 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:15:24.382967    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/45dc2582-a116-4520-9dac-c5d60e2d28b4-kube-api-access-5plz8" (OuterVolumeSpecName: "kube-api-access-5plz8") pod "45dc2582-a116-4520-9dac-c5d60e2d28b4" (UID: "45dc2582-a116-4520-9dac-c5d60e2d28b4"). InnerVolumeSpecName "kube-api-access-5plz8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 13:15:24.383303 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:15:24.383283    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/45dc2582-a116-4520-9dac-c5d60e2d28b4-dshm" (OuterVolumeSpecName: "dshm") pod "45dc2582-a116-4520-9dac-c5d60e2d28b4" (UID: "45dc2582-a116-4520-9dac-c5d60e2d28b4"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 13:15:24.385631 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:15:24.385616    2556 scope.go:117] "RemoveContainer" containerID="231bee3bf39893037b77de23f285326703125f163eb3f3088a1059a309e4891e"
Jul 21 13:15:24.385929 ip-10-0-130-2 kubenswrapper[2556]: E0721 13:15:24.385910    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"231bee3bf39893037b77de23f285326703125f163eb3f3088a1059a309e4891e\": container with ID starting with 231bee3bf39893037b77de23f285326703125f163eb3f3088a1059a309e4891e not found: ID does not exist" containerID="231bee3bf39893037b77de23f285326703125f163eb3f3088a1059a309e4891e"
Jul 21 13:15:24.386047 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:15:24.385938    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"231bee3bf39893037b77de23f285326703125f163eb3f3088a1059a309e4891e"} err="failed to get container status \"231bee3bf39893037b77de23f285326703125f163eb3f3088a1059a309e4891e\": rpc error: code = NotFound desc = could not find container \"231bee3bf39893037b77de23f285326703125f163eb3f3088a1059a309e4891e\": container with ID starting with 231bee3bf39893037b77de23f285326703125f163eb3f3088a1059a309e4891e not found: ID does not exist"
Jul 21 13:15:24.482269 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:15:24.482241    2556 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/45dc2582-a116-4520-9dac-c5d60e2d28b4-tls-certs\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Jul 21 13:15:24.482269 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:15:24.482269    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5plz8\" (UniqueName: \"kubernetes.io/projected/45dc2582-a116-4520-9dac-c5d60e2d28b4-kube-api-access-5plz8\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Jul 21 13:15:24.482415 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:15:24.482283    2556 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/45dc2582-a116-4520-9dac-c5d60e2d28b4-dshm\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Jul 21 13:15:24.482415 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:15:24.482295    2556 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/45dc2582-a116-4520-9dac-c5d60e2d28b4-tmp-dir\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Jul 21 13:15:24.705286 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:15:24.705252    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-4f0dda7a/e2e-lifecycle-model-4f0dda7a-kserve-6445c54499-9tl6c"]
Jul 21 13:15:24.707294 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:15:24.707274    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-4f0dda7a/e2e-lifecycle-model-4f0dda7a-kserve-6445c54499-9tl6c"]
Jul 21 13:15:25.195127 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:15:25.195100    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="45dc2582-a116-4520-9dac-c5d60e2d28b4" path="/var/lib/kubelet/pods/45dc2582-a116-4520-9dac-c5d60e2d28b4/volumes"
Jul 21 13:15:47.403940 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:15:47.403911    2556 scope.go:117] "RemoveContainer" containerID="6db47655fe1d5b014f632cd59f6913c36585a5072d27b67ca70e155e3e03a684"
Jul 21 13:16:47.428571 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:16:47.428540    2556 scope.go:117] "RemoveContainer" containerID="7c68a257ac0b4f2dae45645504cf2c2e2a0f7b6b8396714c3dac8f8fc83e7f0f"
Jul 21 13:16:55.886184 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:16:55.886147    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-360ca4-767bb96554-7n29m"]
Jul 21 13:16:55.886613 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:16:55.886546    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6d1dcc9d-b95a-4412-b460-a17ea6cd0fb0" containerName="revoke-keys"
Jul 21 13:16:55.886613 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:16:55.886558    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="6d1dcc9d-b95a-4412-b460-a17ea6cd0fb0" containerName="revoke-keys"
Jul 21 13:16:55.886613 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:16:55.886579    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="45dc2582-a116-4520-9dac-c5d60e2d28b4" containerName="main"
Jul 21 13:16:55.886613 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:16:55.886585    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="45dc2582-a116-4520-9dac-c5d60e2d28b4" containerName="main"
Jul 21 13:16:55.886745 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:16:55.886659    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="6d1dcc9d-b95a-4412-b460-a17ea6cd0fb0" containerName="revoke-keys"
Jul 21 13:16:55.886745 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:16:55.886668    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="45dc2582-a116-4520-9dac-c5d60e2d28b4" containerName="main"
Jul 21 13:16:55.890080 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:16:55.890055    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-360ca4-767bb96554-7n29m"
Jul 21 13:16:55.892904 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:16:55.892880    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-360ca4-serving-cert\""
Jul 21 13:16:55.905944 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:16:55.905917    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-360ca4-767bb96554-7n29m"]
Jul 21 13:16:56.033820 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:16:56.033781    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p2sp4\" (UniqueName: \"kubernetes.io/projected/ce4454ad-98d3-444f-b285-a2a91cb85c26-kube-api-access-p2sp4\") pod \"maas-api-e2e-inf-a-360ca4-767bb96554-7n29m\" (UID: \"ce4454ad-98d3-444f-b285-a2a91cb85c26\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-360ca4-767bb96554-7n29m"
Jul 21 13:16:56.033983 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:16:56.033831    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ce4454ad-98d3-444f-b285-a2a91cb85c26-openshift-service-ca\") pod \"maas-api-e2e-inf-a-360ca4-767bb96554-7n29m\" (UID: \"ce4454ad-98d3-444f-b285-a2a91cb85c26\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-360ca4-767bb96554-7n29m"
Jul 21 13:16:56.033983 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:16:56.033920    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ce4454ad-98d3-444f-b285-a2a91cb85c26-maas-api-tls\") pod \"maas-api-e2e-inf-a-360ca4-767bb96554-7n29m\" (UID: \"ce4454ad-98d3-444f-b285-a2a91cb85c26\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-360ca4-767bb96554-7n29m"
Jul 21 13:16:56.135351 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:16:56.135322    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p2sp4\" (UniqueName: \"kubernetes.io/projected/ce4454ad-98d3-444f-b285-a2a91cb85c26-kube-api-access-p2sp4\") pod \"maas-api-e2e-inf-a-360ca4-767bb96554-7n29m\" (UID: \"ce4454ad-98d3-444f-b285-a2a91cb85c26\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-360ca4-767bb96554-7n29m"
Jul 21 13:16:56.135507 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:16:56.135364    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ce4454ad-98d3-444f-b285-a2a91cb85c26-openshift-service-ca\") pod \"maas-api-e2e-inf-a-360ca4-767bb96554-7n29m\" (UID: \"ce4454ad-98d3-444f-b285-a2a91cb85c26\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-360ca4-767bb96554-7n29m"
Jul 21 13:16:56.135580 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:16:56.135547    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ce4454ad-98d3-444f-b285-a2a91cb85c26-maas-api-tls\") pod \"maas-api-e2e-inf-a-360ca4-767bb96554-7n29m\" (UID: \"ce4454ad-98d3-444f-b285-a2a91cb85c26\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-360ca4-767bb96554-7n29m"
Jul 21 13:16:56.135701 ip-10-0-130-2 kubenswrapper[2556]: E0721 13:16:56.135683    2556 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-360ca4-serving-cert: secret "maas-api-e2e-inf-a-360ca4-serving-cert" not found
Jul 21 13:16:56.135771 ip-10-0-130-2 kubenswrapper[2556]: E0721 13:16:56.135756    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ce4454ad-98d3-444f-b285-a2a91cb85c26-maas-api-tls podName:ce4454ad-98d3-444f-b285-a2a91cb85c26 nodeName:}" failed. No retries permitted until 2026-07-21 13:16:56.635735081 +0000 UTC m=+2229.939740164 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/ce4454ad-98d3-444f-b285-a2a91cb85c26-maas-api-tls") pod "maas-api-e2e-inf-a-360ca4-767bb96554-7n29m" (UID: "ce4454ad-98d3-444f-b285-a2a91cb85c26") : secret "maas-api-e2e-inf-a-360ca4-serving-cert" not found
Jul 21 13:16:56.136048 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:16:56.135994    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ce4454ad-98d3-444f-b285-a2a91cb85c26-openshift-service-ca\") pod \"maas-api-e2e-inf-a-360ca4-767bb96554-7n29m\" (UID: \"ce4454ad-98d3-444f-b285-a2a91cb85c26\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-360ca4-767bb96554-7n29m"
Jul 21 13:16:56.148715 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:16:56.148665    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p2sp4\" (UniqueName: \"kubernetes.io/projected/ce4454ad-98d3-444f-b285-a2a91cb85c26-kube-api-access-p2sp4\") pod \"maas-api-e2e-inf-a-360ca4-767bb96554-7n29m\" (UID: \"ce4454ad-98d3-444f-b285-a2a91cb85c26\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-360ca4-767bb96554-7n29m"
Jul 21 13:16:56.641525 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:16:56.641494    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ce4454ad-98d3-444f-b285-a2a91cb85c26-maas-api-tls\") pod \"maas-api-e2e-inf-a-360ca4-767bb96554-7n29m\" (UID: \"ce4454ad-98d3-444f-b285-a2a91cb85c26\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-360ca4-767bb96554-7n29m"
Jul 21 13:16:56.644132 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:16:56.644106    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ce4454ad-98d3-444f-b285-a2a91cb85c26-maas-api-tls\") pod \"maas-api-e2e-inf-a-360ca4-767bb96554-7n29m\" (UID: \"ce4454ad-98d3-444f-b285-a2a91cb85c26\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-360ca4-767bb96554-7n29m"
Jul 21 13:16:56.802540 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:16:56.802510    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-360ca4-767bb96554-7n29m"
Jul 21 13:16:56.947694 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:16:56.947671    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-360ca4-767bb96554-7n29m"]
Jul 21 13:16:56.949319 ip-10-0-130-2 kubenswrapper[2556]: W0721 13:16:56.949290    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce4454ad_98d3_444f_b285_a2a91cb85c26.slice/crio-05d8aaa0ad758c4e4380488de35912fb9c1882233b115ca684c67360d01e9dde WatchSource:0}: Error finding container 05d8aaa0ad758c4e4380488de35912fb9c1882233b115ca684c67360d01e9dde: Status 404 returned error can't find the container with id 05d8aaa0ad758c4e4380488de35912fb9c1882233b115ca684c67360d01e9dde
Jul 21 13:16:56.950701 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:16:56.950684    2556 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 21 13:16:57.875571 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:16:57.875537    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-360ca4-767bb96554-7n29m" event={"ID":"ce4454ad-98d3-444f-b285-a2a91cb85c26","Type":"ContainerStarted","Data":"251fb20a888df60bbf0ce79c758d376c5f18382f4c7c525f5868ab399f7347dd"}
Jul 21 13:16:57.875742 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:16:57.875593    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-360ca4-767bb96554-7n29m" event={"ID":"ce4454ad-98d3-444f-b285-a2a91cb85c26","Type":"ContainerStarted","Data":"05d8aaa0ad758c4e4380488de35912fb9c1882233b115ca684c67360d01e9dde"}
Jul 21 13:16:57.875742 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:16:57.875658    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-360ca4-767bb96554-7n29m"
Jul 21 13:16:57.898960 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:16:57.898918    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-360ca4-767bb96554-7n29m" podStartSLOduration=2.456944603 podStartE2EDuration="2.898906715s" totalImagesPullingTime="441.962112ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 13:16:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:16:57.896991483 +0000 UTC m=+2231.200996573" watchObservedRunningTime="2026-07-21 13:16:57.898906715 +0000 UTC m=+2231.202911805"
Jul 21 13:17:03.887227 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:17:03.887201    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-360ca4-767bb96554-7n29m"
Jul 21 13:17:03.918582 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:17:03.918542    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-823a18-798c77c8ff-22qmc"]
Jul 21 13:17:03.927515 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:17:03.927481    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-823a18-798c77c8ff-22qmc"]
Jul 21 13:17:03.928132 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:17:03.927585    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-823a18-798c77c8ff-22qmc"
Jul 21 13:17:03.929887 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:17:03.929863    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-823a18-serving-cert\""
Jul 21 13:17:04.112574 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:17:04.112548    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5ndv8\" (UniqueName: \"kubernetes.io/projected/6246445e-c888-4090-bb26-7d7641a4238a-kube-api-access-5ndv8\") pod \"maas-api-e2e-inf-b-823a18-798c77c8ff-22qmc\" (UID: \"6246445e-c888-4090-bb26-7d7641a4238a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-823a18-798c77c8ff-22qmc"
Jul 21 13:17:04.112741 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:17:04.112613    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6246445e-c888-4090-bb26-7d7641a4238a-openshift-service-ca\") pod \"maas-api-e2e-inf-b-823a18-798c77c8ff-22qmc\" (UID: \"6246445e-c888-4090-bb26-7d7641a4238a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-823a18-798c77c8ff-22qmc"
Jul 21 13:17:04.112741 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:17:04.112648    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6246445e-c888-4090-bb26-7d7641a4238a-maas-api-tls\") pod \"maas-api-e2e-inf-b-823a18-798c77c8ff-22qmc\" (UID: \"6246445e-c888-4090-bb26-7d7641a4238a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-823a18-798c77c8ff-22qmc"
Jul 21 13:17:04.213923 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:17:04.213840    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5ndv8\" (UniqueName: \"kubernetes.io/projected/6246445e-c888-4090-bb26-7d7641a4238a-kube-api-access-5ndv8\") pod \"maas-api-e2e-inf-b-823a18-798c77c8ff-22qmc\" (UID: \"6246445e-c888-4090-bb26-7d7641a4238a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-823a18-798c77c8ff-22qmc"
Jul 21 13:17:04.213923 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:17:04.213911    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6246445e-c888-4090-bb26-7d7641a4238a-openshift-service-ca\") pod \"maas-api-e2e-inf-b-823a18-798c77c8ff-22qmc\" (UID: \"6246445e-c888-4090-bb26-7d7641a4238a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-823a18-798c77c8ff-22qmc"
Jul 21 13:17:04.214183 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:17:04.213931    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6246445e-c888-4090-bb26-7d7641a4238a-maas-api-tls\") pod \"maas-api-e2e-inf-b-823a18-798c77c8ff-22qmc\" (UID: \"6246445e-c888-4090-bb26-7d7641a4238a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-823a18-798c77c8ff-22qmc"
Jul 21 13:17:04.214183 ip-10-0-130-2 kubenswrapper[2556]: E0721 13:17:04.214059    2556 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-823a18-serving-cert: secret "maas-api-e2e-inf-b-823a18-serving-cert" not found
Jul 21 13:17:04.214183 ip-10-0-130-2 kubenswrapper[2556]: E0721 13:17:04.214116    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6246445e-c888-4090-bb26-7d7641a4238a-maas-api-tls podName:6246445e-c888-4090-bb26-7d7641a4238a nodeName:}" failed. No retries permitted until 2026-07-21 13:17:04.714095241 +0000 UTC m=+2238.018100308 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/6246445e-c888-4090-bb26-7d7641a4238a-maas-api-tls") pod "maas-api-e2e-inf-b-823a18-798c77c8ff-22qmc" (UID: "6246445e-c888-4090-bb26-7d7641a4238a") : secret "maas-api-e2e-inf-b-823a18-serving-cert" not found
Jul 21 13:17:04.214564 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:17:04.214545    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6246445e-c888-4090-bb26-7d7641a4238a-openshift-service-ca\") pod \"maas-api-e2e-inf-b-823a18-798c77c8ff-22qmc\" (UID: \"6246445e-c888-4090-bb26-7d7641a4238a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-823a18-798c77c8ff-22qmc"
Jul 21 13:17:04.226969 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:17:04.226947    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5ndv8\" (UniqueName: \"kubernetes.io/projected/6246445e-c888-4090-bb26-7d7641a4238a-kube-api-access-5ndv8\") pod \"maas-api-e2e-inf-b-823a18-798c77c8ff-22qmc\" (UID: \"6246445e-c888-4090-bb26-7d7641a4238a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-823a18-798c77c8ff-22qmc"
Jul 21 13:17:04.717636 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:17:04.717603    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6246445e-c888-4090-bb26-7d7641a4238a-maas-api-tls\") pod \"maas-api-e2e-inf-b-823a18-798c77c8ff-22qmc\" (UID: \"6246445e-c888-4090-bb26-7d7641a4238a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-823a18-798c77c8ff-22qmc"
Jul 21 13:17:04.720155 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:17:04.720135    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6246445e-c888-4090-bb26-7d7641a4238a-maas-api-tls\") pod \"maas-api-e2e-inf-b-823a18-798c77c8ff-22qmc\" (UID: \"6246445e-c888-4090-bb26-7d7641a4238a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-823a18-798c77c8ff-22qmc"
Jul 21 13:17:04.842119 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:17:04.842085    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-823a18-798c77c8ff-22qmc"
Jul 21 13:17:04.983907 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:17:04.983882    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-823a18-798c77c8ff-22qmc"]
Jul 21 13:17:04.985463 ip-10-0-130-2 kubenswrapper[2556]: W0721 13:17:04.985436    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6246445e_c888_4090_bb26_7d7641a4238a.slice/crio-4ed4e3b10a420f2c44ac5a701e6bc7e2bbbed6c52d2b9512000222e860991cc9 WatchSource:0}: Error finding container 4ed4e3b10a420f2c44ac5a701e6bc7e2bbbed6c52d2b9512000222e860991cc9: Status 404 returned error can't find the container with id 4ed4e3b10a420f2c44ac5a701e6bc7e2bbbed6c52d2b9512000222e860991cc9
Jul 21 13:17:05.926391 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:17:05.926351    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-823a18-798c77c8ff-22qmc" event={"ID":"6246445e-c888-4090-bb26-7d7641a4238a","Type":"ContainerStarted","Data":"c70150ffecf8705499f91294adb0dbe1f56b8840560f2a3adaac50970eae11e9"}
Jul 21 13:17:05.926561 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:17:05.926494    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-823a18-798c77c8ff-22qmc"
Jul 21 13:17:05.926561 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:17:05.926530    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-823a18-798c77c8ff-22qmc" event={"ID":"6246445e-c888-4090-bb26-7d7641a4238a","Type":"ContainerStarted","Data":"4ed4e3b10a420f2c44ac5a701e6bc7e2bbbed6c52d2b9512000222e860991cc9"}
Jul 21 13:17:05.959745 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:17:05.959698    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-823a18-798c77c8ff-22qmc" podStartSLOduration=2.618361014 podStartE2EDuration="2.959685742s" totalImagesPullingTime="341.324728ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 13:17:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:17:05.958322366 +0000 UTC m=+2239.262327456" watchObservedRunningTime="2026-07-21 13:17:05.959685742 +0000 UTC m=+2239.263690831"
Jul 21 13:17:11.936636 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:17:11.936562    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-823a18-798c77c8ff-22qmc"
Jul 21 13:17:15.662852 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:17:15.662812    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-360ca4/test-model-360ca4-kserve-596975d795-jdk7m"]
Jul 21 13:17:15.666617 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:17:15.666599    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-360ca4/test-model-360ca4-kserve-596975d795-jdk7m"
Jul 21 13:17:15.670324 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:17:15.670287    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-360ca4\"/\"test-model-360ca4-kserve-self-signed-certs\""
Jul 21 13:17:15.670324 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:17:15.670287    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-360ca4\"/\"kube-root-ca.crt\""
Jul 21 13:17:15.670522 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:17:15.670375    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-360ca4\"/\"openshift-service-ca.crt\""
Jul 21 13:17:15.670522 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:17:15.670319    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-360ca4\"/\"default-dockercfg-vwwpf\""
Jul 21 13:17:15.687699 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:17:15.687570    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-360ca4/test-model-360ca4-kserve-596975d795-jdk7m"]
Jul 21 13:17:15.723082 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:17:15.723045    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ee000558-246c-44c4-a355-24bdd0d06f58-dshm\") pod \"test-model-360ca4-kserve-596975d795-jdk7m\" (UID: \"ee000558-246c-44c4-a355-24bdd0d06f58\") " pod="ai-tenant-e2e-inf-a-360ca4/test-model-360ca4-kserve-596975d795-jdk7m"
Jul 21 13:17:15.723231 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:17:15.723130    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zbpzp\" (UniqueName: \"kubernetes.io/projected/ee000558-246c-44c4-a355-24bdd0d06f58-kube-api-access-zbpzp\") pod \"test-model-360ca4-kserve-596975d795-jdk7m\" (UID: \"ee000558-246c-44c4-a355-24bdd0d06f58\") " pod="ai-tenant-e2e-inf-a-360ca4/test-model-360ca4-kserve-596975d795-jdk7m"
Jul 21 13:17:15.723302 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:17:15.723229    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ee000558-246c-44c4-a355-24bdd0d06f58-model-cache\") pod \"test-model-360ca4-kserve-596975d795-jdk7m\" (UID: \"ee000558-246c-44c4-a355-24bdd0d06f58\") " pod="ai-tenant-e2e-inf-a-360ca4/test-model-360ca4-kserve-596975d795-jdk7m"
Jul 21 13:17:15.723302 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:17:15.723272    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ee000558-246c-44c4-a355-24bdd0d06f58-tmp-dir\") pod \"test-model-360ca4-kserve-596975d795-jdk7m\" (UID: \"ee000558-246c-44c4-a355-24bdd0d06f58\") " pod="ai-tenant-e2e-inf-a-360ca4/test-model-360ca4-kserve-596975d795-jdk7m"
Jul 21 13:17:15.723408 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:17:15.723309    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ee000558-246c-44c4-a355-24bdd0d06f58-home\") pod \"test-model-360ca4-kserve-596975d795-jdk7m\" (UID: \"ee000558-246c-44c4-a355-24bdd0d06f58\") " pod="ai-tenant-e2e-inf-a-360ca4/test-model-360ca4-kserve-596975d795-jdk7m"
Jul 21 13:17:15.723408 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:17:15.723333    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ee000558-246c-44c4-a355-24bdd0d06f58-tls-certs\") pod \"test-model-360ca4-kserve-596975d795-jdk7m\" (UID: \"ee000558-246c-44c4-a355-24bdd0d06f58\") " pod="ai-tenant-e2e-inf-a-360ca4/test-model-360ca4-kserve-596975d795-jdk7m"
Jul 21 13:17:15.824419 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:17:15.824381    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ee000558-246c-44c4-a355-24bdd0d06f58-model-cache\") pod \"test-model-360ca4-kserve-596975d795-jdk7m\" (UID: \"ee000558-246c-44c4-a355-24bdd0d06f58\") " pod="ai-tenant-e2e-inf-a-360ca4/test-model-360ca4-kserve-596975d795-jdk7m"
Jul 21 13:17:15.824622 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:17:15.824428    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ee000558-246c-44c4-a355-24bdd0d06f58-tmp-dir\") pod \"test-model-360ca4-kserve-596975d795-jdk7m\" (UID: \"ee000558-246c-44c4-a355-24bdd0d06f58\") " pod="ai-tenant-e2e-inf-a-360ca4/test-model-360ca4-kserve-596975d795-jdk7m"
Jul 21 13:17:15.824622 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:17:15.824450    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ee000558-246c-44c4-a355-24bdd0d06f58-home\") pod \"test-model-360ca4-kserve-596975d795-jdk7m\" (UID: \"ee000558-246c-44c4-a355-24bdd0d06f58\") " pod="ai-tenant-e2e-inf-a-360ca4/test-model-360ca4-kserve-596975d795-jdk7m"
Jul 21 13:17:15.824622 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:17:15.824466    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ee000558-246c-44c4-a355-24bdd0d06f58-tls-certs\") pod \"test-model-360ca4-kserve-596975d795-jdk7m\" (UID: \"ee000558-246c-44c4-a355-24bdd0d06f58\") " pod="ai-tenant-e2e-inf-a-360ca4/test-model-360ca4-kserve-596975d795-jdk7m"
Jul 21 13:17:15.824622 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:17:15.824505    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ee000558-246c-44c4-a355-24bdd0d06f58-dshm\") pod \"test-model-360ca4-kserve-596975d795-jdk7m\" (UID: \"ee000558-246c-44c4-a355-24bdd0d06f58\") " pod="ai-tenant-e2e-inf-a-360ca4/test-model-360ca4-kserve-596975d795-jdk7m"
Jul 21 13:17:15.824622 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:17:15.824555    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zbpzp\" (UniqueName: \"kubernetes.io/projected/ee000558-246c-44c4-a355-24bdd0d06f58-kube-api-access-zbpzp\") pod \"test-model-360ca4-kserve-596975d795-jdk7m\" (UID: \"ee000558-246c-44c4-a355-24bdd0d06f58\") " pod="ai-tenant-e2e-inf-a-360ca4/test-model-360ca4-kserve-596975d795-jdk7m"
Jul 21 13:17:15.824957 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:17:15.824851    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ee000558-246c-44c4-a355-24bdd0d06f58-home\") pod \"test-model-360ca4-kserve-596975d795-jdk7m\" (UID: \"ee000558-246c-44c4-a355-24bdd0d06f58\") " pod="ai-tenant-e2e-inf-a-360ca4/test-model-360ca4-kserve-596975d795-jdk7m"
Jul 21 13:17:15.824957 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:17:15.824901    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ee000558-246c-44c4-a355-24bdd0d06f58-model-cache\") pod \"test-model-360ca4-kserve-596975d795-jdk7m\" (UID: \"ee000558-246c-44c4-a355-24bdd0d06f58\") " pod="ai-tenant-e2e-inf-a-360ca4/test-model-360ca4-kserve-596975d795-jdk7m"
Jul 21 13:17:15.824957 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:17:15.824939    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ee000558-246c-44c4-a355-24bdd0d06f58-tmp-dir\") pod \"test-model-360ca4-kserve-596975d795-jdk7m\" (UID: \"ee000558-246c-44c4-a355-24bdd0d06f58\") " pod="ai-tenant-e2e-inf-a-360ca4/test-model-360ca4-kserve-596975d795-jdk7m"
Jul 21 13:17:15.826922 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:17:15.826900    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ee000558-246c-44c4-a355-24bdd0d06f58-dshm\") pod \"test-model-360ca4-kserve-596975d795-jdk7m\" (UID: \"ee000558-246c-44c4-a355-24bdd0d06f58\") " pod="ai-tenant-e2e-inf-a-360ca4/test-model-360ca4-kserve-596975d795-jdk7m"
Jul 21 13:17:15.827195 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:17:15.827179    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ee000558-246c-44c4-a355-24bdd0d06f58-tls-certs\") pod \"test-model-360ca4-kserve-596975d795-jdk7m\" (UID: \"ee000558-246c-44c4-a355-24bdd0d06f58\") " pod="ai-tenant-e2e-inf-a-360ca4/test-model-360ca4-kserve-596975d795-jdk7m"
Jul 21 13:17:15.835086 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:17:15.835066    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zbpzp\" (UniqueName: \"kubernetes.io/projected/ee000558-246c-44c4-a355-24bdd0d06f58-kube-api-access-zbpzp\") pod \"test-model-360ca4-kserve-596975d795-jdk7m\" (UID: \"ee000558-246c-44c4-a355-24bdd0d06f58\") " pod="ai-tenant-e2e-inf-a-360ca4/test-model-360ca4-kserve-596975d795-jdk7m"
Jul 21 13:17:15.976963 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:17:15.976937    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-360ca4/test-model-360ca4-kserve-596975d795-jdk7m"
Jul 21 13:17:16.124432 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:17:16.124408    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-360ca4/test-model-360ca4-kserve-596975d795-jdk7m"]
Jul 21 13:17:16.126525 ip-10-0-130-2 kubenswrapper[2556]: W0721 13:17:16.126498    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podee000558_246c_44c4_a355_24bdd0d06f58.slice/crio-df2c9b28cbfab9969f997fc6e59fac3530d9cb4b141192cfeb6f2f9fecbdbdc5 WatchSource:0}: Error finding container df2c9b28cbfab9969f997fc6e59fac3530d9cb4b141192cfeb6f2f9fecbdbdc5: Status 404 returned error can't find the container with id df2c9b28cbfab9969f997fc6e59fac3530d9cb4b141192cfeb6f2f9fecbdbdc5
Jul 21 13:17:16.986944 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:17:16.986902    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-360ca4/test-model-360ca4-kserve-596975d795-jdk7m" event={"ID":"ee000558-246c-44c4-a355-24bdd0d06f58","Type":"ContainerStarted","Data":"e0d58e9889ff3f1ced2cf2a75add0c427267232042ef2c3b64b6ad49db909122"}
Jul 21 13:17:16.987336 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:17:16.986955    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-360ca4/test-model-360ca4-kserve-596975d795-jdk7m" event={"ID":"ee000558-246c-44c4-a355-24bdd0d06f58","Type":"ContainerStarted","Data":"df2c9b28cbfab9969f997fc6e59fac3530d9cb4b141192cfeb6f2f9fecbdbdc5"}
Jul 21 13:17:17.010103 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:17:17.010058    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-360ca4/test-model-360ca4-kserve-596975d795-jdk7m" podStartSLOduration=2.010042651 podStartE2EDuration="2.010042651s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 13:17:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:17:17.009499089 +0000 UTC m=+2250.313504178" watchObservedRunningTime="2026-07-21 13:17:17.010042651 +0000 UTC m=+2250.314047735"
Jul 21 13:17:25.977066 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:17:25.977034    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-360ca4/test-model-360ca4-kserve-596975d795-jdk7m"
Jul 21 13:17:25.977463 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:17:25.977152    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-360ca4/test-model-360ca4-kserve-596975d795-jdk7m"
Jul 21 13:17:25.989415 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:17:25.989389    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-360ca4/test-model-360ca4-kserve-596975d795-jdk7m"
Jul 21 13:17:26.050495 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:17:26.050470    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-360ca4/test-model-360ca4-kserve-596975d795-jdk7m"
Jul 21 13:17:47.448214 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:17:47.448182    2556 scope.go:117] "RemoveContainer" containerID="3d21f31c3275b33db336fdfeaae59e9f56fa99d1ea91bd843ee715708283377a"
Jul 21 13:18:17.856422 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:18:17.856390    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-360ca4/test-model-360ca4-kserve-596975d795-jdk7m"]
Jul 21 13:18:29.640491 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:18:29.640452    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-360ca4-767bb96554-7n29m"]
Jul 21 13:18:29.641034 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:18:29.640703    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-360ca4-767bb96554-7n29m" podUID="ce4454ad-98d3-444f-b285-a2a91cb85c26" containerName="maas-api" containerID="cri-o://251fb20a888df60bbf0ce79c758d376c5f18382f4c7c525f5868ab399f7347dd" gracePeriod=30
Jul 21 13:18:29.897664 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:18:29.897598    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-360ca4-767bb96554-7n29m"
Jul 21 13:18:30.007967 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:18:30.007932    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p2sp4\" (UniqueName: \"kubernetes.io/projected/ce4454ad-98d3-444f-b285-a2a91cb85c26-kube-api-access-p2sp4\") pod \"ce4454ad-98d3-444f-b285-a2a91cb85c26\" (UID: \"ce4454ad-98d3-444f-b285-a2a91cb85c26\") "
Jul 21 13:18:30.008180 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:18:30.008108    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ce4454ad-98d3-444f-b285-a2a91cb85c26-maas-api-tls\") pod \"ce4454ad-98d3-444f-b285-a2a91cb85c26\" (UID: \"ce4454ad-98d3-444f-b285-a2a91cb85c26\") "
Jul 21 13:18:30.008180 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:18:30.008153    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ce4454ad-98d3-444f-b285-a2a91cb85c26-openshift-service-ca\") pod \"ce4454ad-98d3-444f-b285-a2a91cb85c26\" (UID: \"ce4454ad-98d3-444f-b285-a2a91cb85c26\") "
Jul 21 13:18:30.008540 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:18:30.008518    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ce4454ad-98d3-444f-b285-a2a91cb85c26-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ce4454ad-98d3-444f-b285-a2a91cb85c26" (UID: "ce4454ad-98d3-444f-b285-a2a91cb85c26"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 13:18:30.010241 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:18:30.010208    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ce4454ad-98d3-444f-b285-a2a91cb85c26-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ce4454ad-98d3-444f-b285-a2a91cb85c26" (UID: "ce4454ad-98d3-444f-b285-a2a91cb85c26"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 13:18:30.010241 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:18:30.010212    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ce4454ad-98d3-444f-b285-a2a91cb85c26-kube-api-access-p2sp4" (OuterVolumeSpecName: "kube-api-access-p2sp4") pod "ce4454ad-98d3-444f-b285-a2a91cb85c26" (UID: "ce4454ad-98d3-444f-b285-a2a91cb85c26"). InnerVolumeSpecName "kube-api-access-p2sp4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 13:18:30.109850 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:18:30.109820    2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ce4454ad-98d3-444f-b285-a2a91cb85c26-maas-api-tls\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Jul 21 13:18:30.109850 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:18:30.109846    2556 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ce4454ad-98d3-444f-b285-a2a91cb85c26-openshift-service-ca\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Jul 21 13:18:30.109850 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:18:30.109856    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p2sp4\" (UniqueName: \"kubernetes.io/projected/ce4454ad-98d3-444f-b285-a2a91cb85c26-kube-api-access-p2sp4\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Jul 21 13:18:30.397226 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:18:30.397190    2556 generic.go:356] "Generic (PLEG): container finished" podID="ce4454ad-98d3-444f-b285-a2a91cb85c26" containerID="251fb20a888df60bbf0ce79c758d376c5f18382f4c7c525f5868ab399f7347dd" exitCode=0
Jul 21 13:18:30.397393 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:18:30.397238    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-360ca4-767bb96554-7n29m"
Jul 21 13:18:30.397393 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:18:30.397261    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-360ca4-767bb96554-7n29m" event={"ID":"ce4454ad-98d3-444f-b285-a2a91cb85c26","Type":"ContainerDied","Data":"251fb20a888df60bbf0ce79c758d376c5f18382f4c7c525f5868ab399f7347dd"}
Jul 21 13:18:30.397393 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:18:30.397313    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-360ca4-767bb96554-7n29m" event={"ID":"ce4454ad-98d3-444f-b285-a2a91cb85c26","Type":"ContainerDied","Data":"05d8aaa0ad758c4e4380488de35912fb9c1882233b115ca684c67360d01e9dde"}
Jul 21 13:18:30.397393 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:18:30.397342    2556 scope.go:117] "RemoveContainer" containerID="251fb20a888df60bbf0ce79c758d376c5f18382f4c7c525f5868ab399f7347dd"
Jul 21 13:18:30.407297 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:18:30.407276    2556 scope.go:117] "RemoveContainer" containerID="251fb20a888df60bbf0ce79c758d376c5f18382f4c7c525f5868ab399f7347dd"
Jul 21 13:18:30.407562 ip-10-0-130-2 kubenswrapper[2556]: E0721 13:18:30.407542    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"251fb20a888df60bbf0ce79c758d376c5f18382f4c7c525f5868ab399f7347dd\": container with ID starting with 251fb20a888df60bbf0ce79c758d376c5f18382f4c7c525f5868ab399f7347dd not found: ID does not exist" containerID="251fb20a888df60bbf0ce79c758d376c5f18382f4c7c525f5868ab399f7347dd"
Jul 21 13:18:30.407609 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:18:30.407572    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"251fb20a888df60bbf0ce79c758d376c5f18382f4c7c525f5868ab399f7347dd"} err="failed to get container status \"251fb20a888df60bbf0ce79c758d376c5f18382f4c7c525f5868ab399f7347dd\": rpc error: code = NotFound desc = could not find container \"251fb20a888df60bbf0ce79c758d376c5f18382f4c7c525f5868ab399f7347dd\": container with ID starting with 251fb20a888df60bbf0ce79c758d376c5f18382f4c7c525f5868ab399f7347dd not found: ID does not exist"
Jul 21 13:18:30.423911 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:18:30.423882    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-360ca4-767bb96554-7n29m"]
Jul 21 13:18:30.428974 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:18:30.428952    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-360ca4-767bb96554-7n29m"]
Jul 21 13:18:31.194883 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:18:31.194852    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ce4454ad-98d3-444f-b285-a2a91cb85c26" path="/var/lib/kubelet/pods/ce4454ad-98d3-444f-b285-a2a91cb85c26/volumes"
Jul 21 13:18:32.876874 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:18:32.876838    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-360ca4/test-model-360ca4-kserve-596975d795-jdk7m" podUID="ee000558-246c-44c4-a355-24bdd0d06f58" containerName="main" containerID="cri-o://e0d58e9889ff3f1ced2cf2a75add0c427267232042ef2c3b64b6ad49db909122" gracePeriod=45
Jul 21 13:18:33.138534 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:18:33.138465    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-360ca4/test-model-360ca4-kserve-596975d795-jdk7m"
Jul 21 13:18:33.239559 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:18:33.239530    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ee000558-246c-44c4-a355-24bdd0d06f58-dshm\") pod \"ee000558-246c-44c4-a355-24bdd0d06f58\" (UID: \"ee000558-246c-44c4-a355-24bdd0d06f58\") "
Jul 21 13:18:33.239709 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:18:33.239614    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ee000558-246c-44c4-a355-24bdd0d06f58-tls-certs\") pod \"ee000558-246c-44c4-a355-24bdd0d06f58\" (UID: \"ee000558-246c-44c4-a355-24bdd0d06f58\") "
Jul 21 13:18:33.239709 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:18:33.239697    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ee000558-246c-44c4-a355-24bdd0d06f58-tmp-dir\") pod \"ee000558-246c-44c4-a355-24bdd0d06f58\" (UID: \"ee000558-246c-44c4-a355-24bdd0d06f58\") "
Jul 21 13:18:33.239794 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:18:33.239726    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zbpzp\" (UniqueName: \"kubernetes.io/projected/ee000558-246c-44c4-a355-24bdd0d06f58-kube-api-access-zbpzp\") pod \"ee000558-246c-44c4-a355-24bdd0d06f58\" (UID: \"ee000558-246c-44c4-a355-24bdd0d06f58\") "
Jul 21 13:18:33.239794 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:18:33.239752    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ee000558-246c-44c4-a355-24bdd0d06f58-model-cache\") pod \"ee000558-246c-44c4-a355-24bdd0d06f58\" (UID: \"ee000558-246c-44c4-a355-24bdd0d06f58\") "
Jul 21 13:18:33.240034 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:18:33.239878    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ee000558-246c-44c4-a355-24bdd0d06f58-home\") pod \"ee000558-246c-44c4-a355-24bdd0d06f58\" (UID: \"ee000558-246c-44c4-a355-24bdd0d06f58\") "
Jul 21 13:18:33.240034 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:18:33.239976    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ee000558-246c-44c4-a355-24bdd0d06f58-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "ee000558-246c-44c4-a355-24bdd0d06f58" (UID: "ee000558-246c-44c4-a355-24bdd0d06f58"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 13:18:33.240235 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:18:33.240000    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ee000558-246c-44c4-a355-24bdd0d06f58-model-cache" (OuterVolumeSpecName: "model-cache") pod "ee000558-246c-44c4-a355-24bdd0d06f58" (UID: "ee000558-246c-44c4-a355-24bdd0d06f58"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 13:18:33.240235 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:18:33.240145    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ee000558-246c-44c4-a355-24bdd0d06f58-home" (OuterVolumeSpecName: "home") pod "ee000558-246c-44c4-a355-24bdd0d06f58" (UID: "ee000558-246c-44c4-a355-24bdd0d06f58"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 13:18:33.240524 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:18:33.240472    2556 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ee000558-246c-44c4-a355-24bdd0d06f58-tmp-dir\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Jul 21 13:18:33.240524 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:18:33.240492    2556 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ee000558-246c-44c4-a355-24bdd0d06f58-model-cache\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Jul 21 13:18:33.240524 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:18:33.240505    2556 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ee000558-246c-44c4-a355-24bdd0d06f58-home\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Jul 21 13:18:33.241953 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:18:33.241917    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ee000558-246c-44c4-a355-24bdd0d06f58-dshm" (OuterVolumeSpecName: "dshm") pod "ee000558-246c-44c4-a355-24bdd0d06f58" (UID: "ee000558-246c-44c4-a355-24bdd0d06f58"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 13:18:33.241953 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:18:33.241935    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ee000558-246c-44c4-a355-24bdd0d06f58-kube-api-access-zbpzp" (OuterVolumeSpecName: "kube-api-access-zbpzp") pod "ee000558-246c-44c4-a355-24bdd0d06f58" (UID: "ee000558-246c-44c4-a355-24bdd0d06f58"). InnerVolumeSpecName "kube-api-access-zbpzp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 13:18:33.242108 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:18:33.241977    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ee000558-246c-44c4-a355-24bdd0d06f58-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ee000558-246c-44c4-a355-24bdd0d06f58" (UID: "ee000558-246c-44c4-a355-24bdd0d06f58"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 13:18:33.341807 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:18:33.341782    2556 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ee000558-246c-44c4-a355-24bdd0d06f58-dshm\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Jul 21 13:18:33.341807 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:18:33.341806    2556 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ee000558-246c-44c4-a355-24bdd0d06f58-tls-certs\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Jul 21 13:18:33.341964 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:18:33.341822    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zbpzp\" (UniqueName: \"kubernetes.io/projected/ee000558-246c-44c4-a355-24bdd0d06f58-kube-api-access-zbpzp\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Jul 21 13:18:33.416166 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:18:33.416078    2556 generic.go:356] "Generic (PLEG): container finished" podID="ee000558-246c-44c4-a355-24bdd0d06f58" containerID="e0d58e9889ff3f1ced2cf2a75add0c427267232042ef2c3b64b6ad49db909122" exitCode=0
Jul 21 13:18:33.416166 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:18:33.416136    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-360ca4/test-model-360ca4-kserve-596975d795-jdk7m"
Jul 21 13:18:33.416382 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:18:33.416159    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-360ca4/test-model-360ca4-kserve-596975d795-jdk7m" event={"ID":"ee000558-246c-44c4-a355-24bdd0d06f58","Type":"ContainerDied","Data":"e0d58e9889ff3f1ced2cf2a75add0c427267232042ef2c3b64b6ad49db909122"}
Jul 21 13:18:33.416382 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:18:33.416199    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-360ca4/test-model-360ca4-kserve-596975d795-jdk7m" event={"ID":"ee000558-246c-44c4-a355-24bdd0d06f58","Type":"ContainerDied","Data":"df2c9b28cbfab9969f997fc6e59fac3530d9cb4b141192cfeb6f2f9fecbdbdc5"}
Jul 21 13:18:33.416382 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:18:33.416215    2556 scope.go:117] "RemoveContainer" containerID="e0d58e9889ff3f1ced2cf2a75add0c427267232042ef2c3b64b6ad49db909122"
Jul 21 13:18:33.425769 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:18:33.425751    2556 scope.go:117] "RemoveContainer" containerID="e0d58e9889ff3f1ced2cf2a75add0c427267232042ef2c3b64b6ad49db909122"
Jul 21 13:18:33.426075 ip-10-0-130-2 kubenswrapper[2556]: E0721 13:18:33.426057    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e0d58e9889ff3f1ced2cf2a75add0c427267232042ef2c3b64b6ad49db909122\": container with ID starting with e0d58e9889ff3f1ced2cf2a75add0c427267232042ef2c3b64b6ad49db909122 not found: ID does not exist" containerID="e0d58e9889ff3f1ced2cf2a75add0c427267232042ef2c3b64b6ad49db909122"
Jul 21 13:18:33.426123 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:18:33.426085    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e0d58e9889ff3f1ced2cf2a75add0c427267232042ef2c3b64b6ad49db909122"} err="failed to get container status \"e0d58e9889ff3f1ced2cf2a75add0c427267232042ef2c3b64b6ad49db909122\": rpc error: code = NotFound desc = could not find container \"e0d58e9889ff3f1ced2cf2a75add0c427267232042ef2c3b64b6ad49db909122\": container with ID starting with e0d58e9889ff3f1ced2cf2a75add0c427267232042ef2c3b64b6ad49db909122 not found: ID does not exist"
Jul 21 13:18:33.451572 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:18:33.451545    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-360ca4/test-model-360ca4-kserve-596975d795-jdk7m"]
Jul 21 13:18:33.455055 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:18:33.455029    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-360ca4/test-model-360ca4-kserve-596975d795-jdk7m"]
Jul 21 13:18:35.195841 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:18:35.195808    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ee000558-246c-44c4-a355-24bdd0d06f58" path="/var/lib/kubelet/pods/ee000558-246c-44c4-a355-24bdd0d06f58/volumes"
Jul 21 13:18:52.363504 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:18:52.363464    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-823a18-798c77c8ff-22qmc"]
Jul 21 13:18:52.363994 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:18:52.363706    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-823a18-798c77c8ff-22qmc" podUID="6246445e-c888-4090-bb26-7d7641a4238a" containerName="maas-api" containerID="cri-o://c70150ffecf8705499f91294adb0dbe1f56b8840560f2a3adaac50970eae11e9" gracePeriod=30
Jul 21 13:18:52.525954 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:18:52.525922    2556 generic.go:356] "Generic (PLEG): container finished" podID="6246445e-c888-4090-bb26-7d7641a4238a" containerID="c70150ffecf8705499f91294adb0dbe1f56b8840560f2a3adaac50970eae11e9" exitCode=0
Jul 21 13:18:52.526121 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:18:52.525981    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-823a18-798c77c8ff-22qmc" event={"ID":"6246445e-c888-4090-bb26-7d7641a4238a","Type":"ContainerDied","Data":"c70150ffecf8705499f91294adb0dbe1f56b8840560f2a3adaac50970eae11e9"}
Jul 21 13:18:52.629252 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:18:52.629186    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-823a18-798c77c8ff-22qmc"
Jul 21 13:18:52.706141 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:18:52.706109    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6246445e-c888-4090-bb26-7d7641a4238a-openshift-service-ca\") pod \"6246445e-c888-4090-bb26-7d7641a4238a\" (UID: \"6246445e-c888-4090-bb26-7d7641a4238a\") "
Jul 21 13:18:52.706290 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:18:52.706203    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5ndv8\" (UniqueName: \"kubernetes.io/projected/6246445e-c888-4090-bb26-7d7641a4238a-kube-api-access-5ndv8\") pod \"6246445e-c888-4090-bb26-7d7641a4238a\" (UID: \"6246445e-c888-4090-bb26-7d7641a4238a\") "
Jul 21 13:18:52.706290 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:18:52.706264    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6246445e-c888-4090-bb26-7d7641a4238a-maas-api-tls\") pod \"6246445e-c888-4090-bb26-7d7641a4238a\" (UID: \"6246445e-c888-4090-bb26-7d7641a4238a\") "
Jul 21 13:18:52.706446 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:18:52.706424    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6246445e-c888-4090-bb26-7d7641a4238a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "6246445e-c888-4090-bb26-7d7641a4238a" (UID: "6246445e-c888-4090-bb26-7d7641a4238a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 13:18:52.706545 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:18:52.706530    2556 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6246445e-c888-4090-bb26-7d7641a4238a-openshift-service-ca\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Jul 21 13:18:52.708397 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:18:52.708361    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6246445e-c888-4090-bb26-7d7641a4238a-kube-api-access-5ndv8" (OuterVolumeSpecName: "kube-api-access-5ndv8") pod "6246445e-c888-4090-bb26-7d7641a4238a" (UID: "6246445e-c888-4090-bb26-7d7641a4238a"). InnerVolumeSpecName "kube-api-access-5ndv8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 13:18:52.708397 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:18:52.708377    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6246445e-c888-4090-bb26-7d7641a4238a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "6246445e-c888-4090-bb26-7d7641a4238a" (UID: "6246445e-c888-4090-bb26-7d7641a4238a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 13:18:52.807033 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:18:52.806982    2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6246445e-c888-4090-bb26-7d7641a4238a-maas-api-tls\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Jul 21 13:18:52.807033 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:18:52.807005    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5ndv8\" (UniqueName: \"kubernetes.io/projected/6246445e-c888-4090-bb26-7d7641a4238a-kube-api-access-5ndv8\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Jul 21 13:18:53.533146 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:18:53.533108    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-823a18-798c77c8ff-22qmc"
Jul 21 13:18:53.533578 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:18:53.533151    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-823a18-798c77c8ff-22qmc" event={"ID":"6246445e-c888-4090-bb26-7d7641a4238a","Type":"ContainerDied","Data":"4ed4e3b10a420f2c44ac5a701e6bc7e2bbbed6c52d2b9512000222e860991cc9"}
Jul 21 13:18:53.533578 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:18:53.533219    2556 scope.go:117] "RemoveContainer" containerID="c70150ffecf8705499f91294adb0dbe1f56b8840560f2a3adaac50970eae11e9"
Jul 21 13:18:53.556334 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:18:53.556304    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-823a18-798c77c8ff-22qmc"]
Jul 21 13:18:53.561779 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:18:53.561755    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-823a18-798c77c8ff-22qmc"]
Jul 21 13:18:55.195207 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:18:55.195173    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6246445e-c888-4090-bb26-7d7641a4238a" path="/var/lib/kubelet/pods/6246445e-c888-4090-bb26-7d7641a4238a/volumes"
Jul 21 13:19:05.239525 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:19:05.239495    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-d921bc-openshift-default-84d4d4f9b5-bnvgs"]
Jul 21 13:19:05.239963 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:19:05.239918    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ce4454ad-98d3-444f-b285-a2a91cb85c26" containerName="maas-api"
Jul 21 13:19:05.239963 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:19:05.239930    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="ce4454ad-98d3-444f-b285-a2a91cb85c26" containerName="maas-api"
Jul 21 13:19:05.239963 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:19:05.239948    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee000558-246c-44c4-a355-24bdd0d06f58" containerName="main"
Jul 21 13:19:05.239963 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:19:05.239953    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee000558-246c-44c4-a355-24bdd0d06f58" containerName="main"
Jul 21 13:19:05.240121 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:19:05.239968    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6246445e-c888-4090-bb26-7d7641a4238a" containerName="maas-api"
Jul 21 13:19:05.240121 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:19:05.239973    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="6246445e-c888-4090-bb26-7d7641a4238a" containerName="maas-api"
Jul 21 13:19:05.240121 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:19:05.240062    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="ee000558-246c-44c4-a355-24bdd0d06f58" containerName="main"
Jul 21 13:19:05.240121 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:19:05.240071    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="ce4454ad-98d3-444f-b285-a2a91cb85c26" containerName="maas-api"
Jul 21 13:19:05.240121 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:19:05.240078    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="6246445e-c888-4090-bb26-7d7641a4238a" containerName="maas-api"
Jul 21 13:19:05.244465 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:19:05.244448    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-d921bc-openshift-default-84d4d4f9b5-bnvgs"
Jul 21 13:19:05.248306 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:19:05.248285    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-d921bc-openshift-default-dockercfg-pmpl9\""
Jul 21 13:19:05.274625 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:19:05.274604    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-d921bc-openshift-default-84d4d4f9b5-bnvgs"]
Jul 21 13:19:05.310516 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:19:05.310489    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f7fe0978-0f99-4e5d-bbf3-32e8cd0610f7-istio-envoy\") pod \"e2e-api-a-d921bc-openshift-default-84d4d4f9b5-bnvgs\" (UID: \"f7fe0978-0f99-4e5d-bbf3-32e8cd0610f7\") " pod="openshift-ingress/e2e-api-a-d921bc-openshift-default-84d4d4f9b5-bnvgs"
Jul 21 13:19:05.310633 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:19:05.310529    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f7fe0978-0f99-4e5d-bbf3-32e8cd0610f7-istio-token\") pod \"e2e-api-a-d921bc-openshift-default-84d4d4f9b5-bnvgs\" (UID: \"f7fe0978-0f99-4e5d-bbf3-32e8cd0610f7\") " pod="openshift-ingress/e2e-api-a-d921bc-openshift-default-84d4d4f9b5-bnvgs"
Jul 21 13:19:05.310633 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:19:05.310556    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f7fe0978-0f99-4e5d-bbf3-32e8cd0610f7-istiod-ca-cert\") pod \"e2e-api-a-d921bc-openshift-default-84d4d4f9b5-bnvgs\" (UID: \"f7fe0978-0f99-4e5d-bbf3-32e8cd0610f7\") " pod="openshift-ingress/e2e-api-a-d921bc-openshift-default-84d4d4f9b5-bnvgs"
Jul 21 13:19:05.310721 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:19:05.310647    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f7fe0978-0f99-4e5d-bbf3-32e8cd0610f7-workload-certs\") pod \"e2e-api-a-d921bc-openshift-default-84d4d4f9b5-bnvgs\" (UID: \"f7fe0978-0f99-4e5d-bbf3-32e8cd0610f7\") " pod="openshift-ingress/e2e-api-a-d921bc-openshift-default-84d4d4f9b5-bnvgs"
Jul 21 13:19:05.310774 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:19:05.310748    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7fe0978-0f99-4e5d-bbf3-32e8cd0610f7-workload-socket\") pod \"e2e-api-a-d921bc-openshift-default-84d4d4f9b5-bnvgs\" (UID: \"f7fe0978-0f99-4e5d-bbf3-32e8cd0610f7\") " pod="openshift-ingress/e2e-api-a-d921bc-openshift-default-84d4d4f9b5-bnvgs"
Jul 21 13:19:05.310814 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:19:05.310798    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7fe0978-0f99-4e5d-bbf3-32e8cd0610f7-credential-socket\") pod \"e2e-api-a-d921bc-openshift-default-84d4d4f9b5-bnvgs\" (UID: \"f7fe0978-0f99-4e5d-bbf3-32e8cd0610f7\") " pod="openshift-ingress/e2e-api-a-d921bc-openshift-default-84d4d4f9b5-bnvgs"
Jul 21 13:19:05.310850 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:19:05.310825    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f7fe0978-0f99-4e5d-bbf3-32e8cd0610f7-istio-data\") pod \"e2e-api-a-d921bc-openshift-default-84d4d4f9b5-bnvgs\" (UID: \"f7fe0978-0f99-4e5d-bbf3-32e8cd0610f7\") " pod="openshift-ingress/e2e-api-a-d921bc-openshift-default-84d4d4f9b5-bnvgs"
Jul 21 13:19:05.310893 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:19:05.310846    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f7fe0978-0f99-4e5d-bbf3-32e8cd0610f7-istio-podinfo\") pod \"e2e-api-a-d921bc-openshift-default-84d4d4f9b5-bnvgs\" (UID: \"f7fe0978-0f99-4e5d-bbf3-32e8cd0610f7\") " pod="openshift-ingress/e2e-api-a-d921bc-openshift-default-84d4d4f9b5-bnvgs"
Jul 21 13:19:05.310938 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:19:05.310902    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zkp4w\" (UniqueName: \"kubernetes.io/projected/f7fe0978-0f99-4e5d-bbf3-32e8cd0610f7-kube-api-access-zkp4w\") pod \"e2e-api-a-d921bc-openshift-default-84d4d4f9b5-bnvgs\" (UID: \"f7fe0978-0f99-4e5d-bbf3-32e8cd0610f7\") " pod="openshift-ingress/e2e-api-a-d921bc-openshift-default-84d4d4f9b5-bnvgs"
Jul 21 13:19:05.411664 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:19:05.411634    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f7fe0978-0f99-4e5d-bbf3-32e8cd0610f7-istio-envoy\") pod \"e2e-api-a-d921bc-openshift-default-84d4d4f9b5-bnvgs\" (UID: \"f7fe0978-0f99-4e5d-bbf3-32e8cd0610f7\") " pod="openshift-ingress/e2e-api-a-d921bc-openshift-default-84d4d4f9b5-bnvgs"
Jul 21 13:19:05.411664 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:19:05.411670    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f7fe0978-0f99-4e5d-bbf3-32e8cd0610f7-istio-token\") pod \"e2e-api-a-d921bc-openshift-default-84d4d4f9b5-bnvgs\" (UID: \"f7fe0978-0f99-4e5d-bbf3-32e8cd0610f7\") " pod="openshift-ingress/e2e-api-a-d921bc-openshift-default-84d4d4f9b5-bnvgs"
Jul 21 13:19:05.411878 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:19:05.411687    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f7fe0978-0f99-4e5d-bbf3-32e8cd0610f7-istiod-ca-cert\") pod \"e2e-api-a-d921bc-openshift-default-84d4d4f9b5-bnvgs\" (UID: \"f7fe0978-0f99-4e5d-bbf3-32e8cd0610f7\") " pod="openshift-ingress/e2e-api-a-d921bc-openshift-default-84d4d4f9b5-bnvgs"
Jul 21 13:19:05.411878 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:19:05.411711    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f7fe0978-0f99-4e5d-bbf3-32e8cd0610f7-workload-certs\") pod \"e2e-api-a-d921bc-openshift-default-84d4d4f9b5-bnvgs\" (UID: \"f7fe0978-0f99-4e5d-bbf3-32e8cd0610f7\") " pod="openshift-ingress/e2e-api-a-d921bc-openshift-default-84d4d4f9b5-bnvgs"
Jul 21 13:19:05.411878 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:19:05.411764    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7fe0978-0f99-4e5d-bbf3-32e8cd0610f7-workload-socket\") pod \"e2e-api-a-d921bc-openshift-default-84d4d4f9b5-bnvgs\" (UID: \"f7fe0978-0f99-4e5d-bbf3-32e8cd0610f7\") " pod="openshift-ingress/e2e-api-a-d921bc-openshift-default-84d4d4f9b5-bnvgs"
Jul 21 13:19:05.411878 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:19:05.411791    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7fe0978-0f99-4e5d-bbf3-32e8cd0610f7-credential-socket\") pod \"e2e-api-a-d921bc-openshift-default-84d4d4f9b5-bnvgs\" (UID: \"f7fe0978-0f99-4e5d-bbf3-32e8cd0610f7\") " pod="openshift-ingress/e2e-api-a-d921bc-openshift-default-84d4d4f9b5-bnvgs"
Jul 21 13:19:05.411878 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:19:05.411813    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f7fe0978-0f99-4e5d-bbf3-32e8cd0610f7-istio-data\") pod \"e2e-api-a-d921bc-openshift-default-84d4d4f9b5-bnvgs\" (UID: \"f7fe0978-0f99-4e5d-bbf3-32e8cd0610f7\") " pod="openshift-ingress/e2e-api-a-d921bc-openshift-default-84d4d4f9b5-bnvgs"
Jul 21 13:19:05.411878 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:19:05.411843    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f7fe0978-0f99-4e5d-bbf3-32e8cd0610f7-istio-podinfo\") pod \"e2e-api-a-d921bc-openshift-default-84d4d4f9b5-bnvgs\" (UID: \"f7fe0978-0f99-4e5d-bbf3-32e8cd0610f7\") " pod="openshift-ingress/e2e-api-a-d921bc-openshift-default-84d4d4f9b5-bnvgs"
Jul 21 13:19:05.411878 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:19:05.411875    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zkp4w\" (UniqueName: \"kubernetes.io/projected/f7fe0978-0f99-4e5d-bbf3-32e8cd0610f7-kube-api-access-zkp4w\") pod \"e2e-api-a-d921bc-openshift-default-84d4d4f9b5-bnvgs\" (UID: \"f7fe0978-0f99-4e5d-bbf3-32e8cd0610f7\") " pod="openshift-ingress/e2e-api-a-d921bc-openshift-default-84d4d4f9b5-bnvgs"
Jul 21 13:19:05.412271 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:19:05.412173    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7fe0978-0f99-4e5d-bbf3-32e8cd0610f7-credential-socket\") pod \"e2e-api-a-d921bc-openshift-default-84d4d4f9b5-bnvgs\" (UID: \"f7fe0978-0f99-4e5d-bbf3-32e8cd0610f7\") " pod="openshift-ingress/e2e-api-a-d921bc-openshift-default-84d4d4f9b5-bnvgs"
Jul 21 13:19:05.412330 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:19:05.412268    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f7fe0978-0f99-4e5d-bbf3-32e8cd0610f7-istio-data\") pod \"e2e-api-a-d921bc-openshift-default-84d4d4f9b5-bnvgs\" (UID: \"f7fe0978-0f99-4e5d-bbf3-32e8cd0610f7\") " pod="openshift-ingress/e2e-api-a-d921bc-openshift-default-84d4d4f9b5-bnvgs"
Jul 21 13:19:05.412391 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:19:05.412347    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7fe0978-0f99-4e5d-bbf3-32e8cd0610f7-workload-socket\") pod \"e2e-api-a-d921bc-openshift-default-84d4d4f9b5-bnvgs\" (UID: \"f7fe0978-0f99-4e5d-bbf3-32e8cd0610f7\") " pod="openshift-ingress/e2e-api-a-d921bc-openshift-default-84d4d4f9b5-bnvgs"
Jul 21 13:19:05.412453 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:19:05.412392    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f7fe0978-0f99-4e5d-bbf3-32e8cd0610f7-workload-certs\") pod \"e2e-api-a-d921bc-openshift-default-84d4d4f9b5-bnvgs\" (UID: \"f7fe0978-0f99-4e5d-bbf3-32e8cd0610f7\") " pod="openshift-ingress/e2e-api-a-d921bc-openshift-default-84d4d4f9b5-bnvgs"
Jul 21 13:19:05.412525 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:19:05.412507    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f7fe0978-0f99-4e5d-bbf3-32e8cd0610f7-istiod-ca-cert\") pod \"e2e-api-a-d921bc-openshift-default-84d4d4f9b5-bnvgs\" (UID: \"f7fe0978-0f99-4e5d-bbf3-32e8cd0610f7\") " pod="openshift-ingress/e2e-api-a-d921bc-openshift-default-84d4d4f9b5-bnvgs"
Jul 21 13:19:05.413916 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:19:05.413898    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f7fe0978-0f99-4e5d-bbf3-32e8cd0610f7-istio-envoy\") pod \"e2e-api-a-d921bc-openshift-default-84d4d4f9b5-bnvgs\" (UID: \"f7fe0978-0f99-4e5d-bbf3-32e8cd0610f7\") " pod="openshift-ingress/e2e-api-a-d921bc-openshift-default-84d4d4f9b5-bnvgs"
Jul 21 13:19:05.414405 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:19:05.414382    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f7fe0978-0f99-4e5d-bbf3-32e8cd0610f7-istio-podinfo\") pod \"e2e-api-a-d921bc-openshift-default-84d4d4f9b5-bnvgs\" (UID: \"f7fe0978-0f99-4e5d-bbf3-32e8cd0610f7\") " pod="openshift-ingress/e2e-api-a-d921bc-openshift-default-84d4d4f9b5-bnvgs"
Jul 21 13:19:05.422899 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:19:05.422877    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zkp4w\" (UniqueName: \"kubernetes.io/projected/f7fe0978-0f99-4e5d-bbf3-32e8cd0610f7-kube-api-access-zkp4w\") pod \"e2e-api-a-d921bc-openshift-default-84d4d4f9b5-bnvgs\" (UID: \"f7fe0978-0f99-4e5d-bbf3-32e8cd0610f7\") " pod="openshift-ingress/e2e-api-a-d921bc-openshift-default-84d4d4f9b5-bnvgs"
Jul 21 13:19:05.424369 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:19:05.424348    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f7fe0978-0f99-4e5d-bbf3-32e8cd0610f7-istio-token\") pod \"e2e-api-a-d921bc-openshift-default-84d4d4f9b5-bnvgs\" (UID: \"f7fe0978-0f99-4e5d-bbf3-32e8cd0610f7\") " pod="openshift-ingress/e2e-api-a-d921bc-openshift-default-84d4d4f9b5-bnvgs"
Jul 21 13:19:05.554978 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:19:05.554922    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-d921bc-openshift-default-84d4d4f9b5-bnvgs"
Jul 21 13:19:05.724386 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:19:05.724321    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-d921bc-openshift-default-84d4d4f9b5-bnvgs"]
Jul 21 13:19:05.727164 ip-10-0-130-2 kubenswrapper[2556]: W0721 13:19:05.727135    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7fe0978_0f99_4e5d_bbf3_32e8cd0610f7.slice/crio-2362804ca25c884820859c26b8446b427a35a07b3b890ac7ff83520911cfc9ab WatchSource:0}: Error finding container 2362804ca25c884820859c26b8446b427a35a07b3b890ac7ff83520911cfc9ab: Status 404 returned error can't find the container with id 2362804ca25c884820859c26b8446b427a35a07b3b890ac7ff83520911cfc9ab
Jul 21 13:19:05.729265 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:19:05.729234    2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 21 13:19:05.729347 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:19:05.729299    2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 21 13:19:05.729347 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:19:05.729328    2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 21 13:19:06.608329 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:19:06.608294    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-d921bc-openshift-default-84d4d4f9b5-bnvgs" event={"ID":"f7fe0978-0f99-4e5d-bbf3-32e8cd0610f7","Type":"ContainerStarted","Data":"08f201cb6d5b625cafb3760549cc6a1b11d38dd4167a7ad07b00e08fcb7ba0e4"}
Jul 21 13:19:06.608781 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:19:06.608351    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-d921bc-openshift-default-84d4d4f9b5-bnvgs" event={"ID":"f7fe0978-0f99-4e5d-bbf3-32e8cd0610f7","Type":"ContainerStarted","Data":"2362804ca25c884820859c26b8446b427a35a07b3b890ac7ff83520911cfc9ab"}
Jul 21 13:19:06.633632 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:19:06.633584    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-d921bc-openshift-default-84d4d4f9b5-bnvgs" podStartSLOduration=1.633566721 podStartE2EDuration="1.633566721s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 13:19:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:19:06.630329724 +0000 UTC m=+2359.934334815" watchObservedRunningTime="2026-07-21 13:19:06.633566721 +0000 UTC m=+2359.937571812"
Jul 21 13:19:07.555360 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:19:07.555325    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-d921bc-openshift-default-84d4d4f9b5-bnvgs"
Jul 21 13:19:07.560108 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:19:07.560085    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-d921bc-openshift-default-84d4d4f9b5-bnvgs"
Jul 21 13:19:07.613618 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:19:07.613594    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-d921bc-openshift-default-84d4d4f9b5-bnvgs"
Jul 21 13:19:07.614443 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:19:07.614427    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-d921bc-openshift-default-84d4d4f9b5-bnvgs"
Jul 21 13:19:22.615378 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:19:22.615304    2556 patch_prober.go:28] interesting pod/e2e-api-a-d921bc-openshift-default-84d4d4f9b5-bnvgs container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body=
Jul 21 13:19:22.615378 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:19:22.615362    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-d921bc-openshift-default-84d4d4f9b5-bnvgs" podUID="f7fe0978-0f99-4e5d-bbf3-32e8cd0610f7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused"
Jul 21 13:19:22.700859 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:19:22.700832    2556 generic.go:356] "Generic (PLEG): container finished" podID="f7fe0978-0f99-4e5d-bbf3-32e8cd0610f7" containerID="08f201cb6d5b625cafb3760549cc6a1b11d38dd4167a7ad07b00e08fcb7ba0e4" exitCode=0
Jul 21 13:19:22.700986 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:19:22.700906    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-d921bc-openshift-default-84d4d4f9b5-bnvgs" event={"ID":"f7fe0978-0f99-4e5d-bbf3-32e8cd0610f7","Type":"ContainerDied","Data":"08f201cb6d5b625cafb3760549cc6a1b11d38dd4167a7ad07b00e08fcb7ba0e4"}
Jul 21 13:19:22.701313 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:19:22.701297    2556 scope.go:117] "RemoveContainer" containerID="08f201cb6d5b625cafb3760549cc6a1b11d38dd4167a7ad07b00e08fcb7ba0e4"
Jul 21 13:19:22.702390 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:19:22.702361    2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 21 13:19:22.702486 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:19:22.702469    2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 21 13:19:22.702524 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:19:22.702512    2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 21 13:19:23.712483 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:19:23.712438    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-d921bc-openshift-default-84d4d4f9b5-bnvgs" event={"ID":"f7fe0978-0f99-4e5d-bbf3-32e8cd0610f7","Type":"ContainerStarted","Data":"3f1b152bb3124562b7d500de742e28b40280d9726ab1ca6227f196e06854e0de"}
Jul 21 13:19:24.555505 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:19:24.555466    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-d921bc-openshift-default-84d4d4f9b5-bnvgs"
Jul 21 13:19:24.555682 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:19:24.555657    2556 patch_prober.go:28] interesting pod/e2e-api-a-d921bc-openshift-default-84d4d4f9b5-bnvgs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body=
Jul 21 13:19:24.555731 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:19:24.555712    2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-d921bc-openshift-default-84d4d4f9b5-bnvgs" podUID="f7fe0978-0f99-4e5d-bbf3-32e8cd0610f7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused"
Jul 21 13:19:25.555351 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:19:25.555316    2556 patch_prober.go:28] interesting pod/e2e-api-a-d921bc-openshift-default-84d4d4f9b5-bnvgs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body=
Jul 21 13:19:25.555733 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:19:25.555378    2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-d921bc-openshift-default-84d4d4f9b5-bnvgs" podUID="f7fe0978-0f99-4e5d-bbf3-32e8cd0610f7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused"
Jul 21 13:19:26.555525 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:19:26.555491    2556 patch_prober.go:28] interesting pod/e2e-api-a-d921bc-openshift-default-84d4d4f9b5-bnvgs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body=
Jul 21 13:19:26.555901 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:19:26.555550    2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-d921bc-openshift-default-84d4d4f9b5-bnvgs" podUID="f7fe0978-0f99-4e5d-bbf3-32e8cd0610f7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused"
Jul 21 13:19:27.555422 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:19:27.555390    2556 patch_prober.go:28] interesting pod/e2e-api-a-d921bc-openshift-default-84d4d4f9b5-bnvgs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body=
Jul 21 13:19:27.555604 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:19:27.555446    2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-d921bc-openshift-default-84d4d4f9b5-bnvgs" podUID="f7fe0978-0f99-4e5d-bbf3-32e8cd0610f7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused"
Jul 21 13:19:28.560232 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:19:28.560198    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-d921bc-openshift-default-84d4d4f9b5-bnvgs"
Jul 21 13:19:28.560712 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:19:28.560675    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-d921bc-openshift-default-84d4d4f9b5-bnvgs"
Jul 21 13:19:28.561537 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:19:28.561513    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-d921bc-openshift-default-84d4d4f9b5-bnvgs"
Jul 21 13:19:30.237855 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:19:30.237819    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-d921bc-gb99h"]
Jul 21 13:19:30.242241 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:19:30.242217    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-d921bc-gb99h"
Jul 21 13:19:30.244860 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:19:30.244840    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5nsrr\""
Jul 21 13:19:30.252487 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:19:30.252463    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-d921bc-gb99h"]
Jul 21 13:19:30.347726 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:19:30.347701    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b736e362-058c-4241-9c8c-5d7a8342588d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-d921bc-gb99h\" (UID: \"b736e362-058c-4241-9c8c-5d7a8342588d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-d921bc-gb99h"
Jul 21 13:19:30.448141 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:19:30.448113    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b736e362-058c-4241-9c8c-5d7a8342588d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-d921bc-gb99h\" (UID: \"b736e362-058c-4241-9c8c-5d7a8342588d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-d921bc-gb99h"
Jul 21 13:19:30.448692 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:19:30.448669    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b736e362-058c-4241-9c8c-5d7a8342588d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-d921bc-gb99h\" (UID: \"b736e362-058c-4241-9c8c-5d7a8342588d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-d921bc-gb99h"
Jul 21 13:19:30.554354 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:19:30.554305    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-d921bc-gb99h"
Jul 21 13:19:30.709438 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:19:30.709413    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-d921bc-gb99h"]
Jul 21 13:19:30.711628 ip-10-0-130-2 kubenswrapper[2556]: W0721 13:19:30.711596    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb736e362_058c_4241_9c8c_5d7a8342588d.slice/crio-2914516b9dde225aed37b6571efc9ffb1e12dd77130f518207f9769691a09246 WatchSource:0}: Error finding container 2914516b9dde225aed37b6571efc9ffb1e12dd77130f518207f9769691a09246: Status 404 returned error can't find the container with id 2914516b9dde225aed37b6571efc9ffb1e12dd77130f518207f9769691a09246
Jul 21 13:19:30.751800 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:19:30.751773    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-d921bc-gb99h" event={"ID":"b736e362-058c-4241-9c8c-5d7a8342588d","Type":"ContainerStarted","Data":"2914516b9dde225aed37b6571efc9ffb1e12dd77130f518207f9769691a09246"}
Jul 21 13:19:31.759241 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:19:31.759212    2556 generic.go:356] "Generic (PLEG): container finished" podID="b736e362-058c-4241-9c8c-5d7a8342588d" containerID="9672be3e6498ab2e03c8b6681ccaf6c945d62129d17733778dfbdca7c5c55227" exitCode=0
Jul 21 13:19:31.759664 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:19:31.759251    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-d921bc-gb99h" event={"ID":"b736e362-058c-4241-9c8c-5d7a8342588d","Type":"ContainerDied","Data":"9672be3e6498ab2e03c8b6681ccaf6c945d62129d17733778dfbdca7c5c55227"}
Jul 21 13:19:32.905701 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:19:32.905677    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-d921bc-gb99h"
Jul 21 13:19:32.969451 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:19:32.969424    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b736e362-058c-4241-9c8c-5d7a8342588d-maas-api-service-ca\") pod \"b736e362-058c-4241-9c8c-5d7a8342588d\" (UID: \"b736e362-058c-4241-9c8c-5d7a8342588d\") "
Jul 21 13:19:32.969745 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:19:32.969720    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b736e362-058c-4241-9c8c-5d7a8342588d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "b736e362-058c-4241-9c8c-5d7a8342588d" (UID: "b736e362-058c-4241-9c8c-5d7a8342588d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 13:19:33.070954 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:19:33.070892    2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b736e362-058c-4241-9c8c-5d7a8342588d-maas-api-service-ca\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Jul 21 13:19:33.772279 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:19:33.772249    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-d921bc-gb99h"
Jul 21 13:19:33.772440 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:19:33.772287    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-d921bc-gb99h" event={"ID":"b736e362-058c-4241-9c8c-5d7a8342588d","Type":"ContainerDied","Data":"2914516b9dde225aed37b6571efc9ffb1e12dd77130f518207f9769691a09246"}
Jul 21 13:19:33.772440 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:19:33.772341    2556 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2914516b9dde225aed37b6571efc9ffb1e12dd77130f518207f9769691a09246"
Jul 21 13:19:47.485529 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:19:47.485498    2556 scope.go:117] "RemoveContainer" containerID="4eec062d8115e0bd372a0e051f2d5e4e2d01a7a72ba8769a14b7008fa075d55e"
Jul 21 13:19:50.338147 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:19:50.338117    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-d921bc-gb99h"]
Jul 21 13:19:50.341782 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:19:50.341758    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-d921bc-gb99h"]
Jul 21 13:19:51.194760 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:19:51.194729    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b736e362-058c-4241-9c8c-5d7a8342588d" path="/var/lib/kubelet/pods/b736e362-058c-4241-9c8c-5d7a8342588d/volumes"
Jul 21 13:19:57.983097 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:19:57.983066    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-d921bc-openshift-default-84d4d4f9b5-bnvgs"]
Jul 21 13:19:57.983513 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:19:57.983276    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-d921bc-openshift-default-84d4d4f9b5-bnvgs" podUID="f7fe0978-0f99-4e5d-bbf3-32e8cd0610f7" containerName="istio-proxy" containerID="cri-o://3f1b152bb3124562b7d500de742e28b40280d9726ab1ca6227f196e06854e0de" gracePeriod=30
Jul 21 13:19:58.377304 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:19:58.377207    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-04ee34-sbjsx"]
Jul 21 13:19:58.377673 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:19:58.377656    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b736e362-058c-4241-9c8c-5d7a8342588d" containerName="revoke-keys"
Jul 21 13:19:58.377733 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:19:58.377673    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="b736e362-058c-4241-9c8c-5d7a8342588d" containerName="revoke-keys"
Jul 21 13:19:58.377807 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:19:58.377796    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="b736e362-058c-4241-9c8c-5d7a8342588d" containerName="revoke-keys"
Jul 21 13:19:58.380827 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:19:58.380811    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-04ee34-sbjsx"
Jul 21 13:19:58.384397 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:19:58.384374    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5nsrr\""
Jul 21 13:19:58.391876 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:19:58.391858    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-04ee34-sbjsx"]
Jul 21 13:19:58.484326 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:19:58.484294    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/963fe765-b48e-484a-9904-567374393136-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-04ee34-sbjsx\" (UID: \"963fe765-b48e-484a-9904-567374393136\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-04ee34-sbjsx"
Jul 21 13:19:58.563191 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:19:58.563163    2556 patch_prober.go:28] interesting pod/e2e-api-a-d921bc-openshift-default-84d4d4f9b5-bnvgs container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 21 13:19:58.563285 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:19:58.563207    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-d921bc-openshift-default-84d4d4f9b5-bnvgs" podUID="f7fe0978-0f99-4e5d-bbf3-32e8cd0610f7" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 21 13:19:58.585475 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:19:58.585452    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/963fe765-b48e-484a-9904-567374393136-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-04ee34-sbjsx\" (UID: \"963fe765-b48e-484a-9904-567374393136\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-04ee34-sbjsx"
Jul 21 13:19:58.586069 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:19:58.586050    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/963fe765-b48e-484a-9904-567374393136-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-04ee34-sbjsx\" (UID: \"963fe765-b48e-484a-9904-567374393136\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-04ee34-sbjsx"
Jul 21 13:19:58.692458 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:19:58.692398    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-04ee34-sbjsx"
Jul 21 13:19:58.874073 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:19:58.874049    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-04ee34-sbjsx"]
Jul 21 13:19:58.876344 ip-10-0-130-2 kubenswrapper[2556]: W0721 13:19:58.876313    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod963fe765_b48e_484a_9904_567374393136.slice/crio-82b3ae178c94420eda83090b0a3e660f4b4f820ce39ad0f1949f62ad874609e9 WatchSource:0}: Error finding container 82b3ae178c94420eda83090b0a3e660f4b4f820ce39ad0f1949f62ad874609e9: Status 404 returned error can't find the container with id 82b3ae178c94420eda83090b0a3e660f4b4f820ce39ad0f1949f62ad874609e9
Jul 21 13:19:58.908645 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:19:58.908620    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-04ee34-sbjsx" event={"ID":"963fe765-b48e-484a-9904-567374393136","Type":"ContainerStarted","Data":"82b3ae178c94420eda83090b0a3e660f4b4f820ce39ad0f1949f62ad874609e9"}
Jul 21 13:19:59.915420 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:19:59.915357    2556 generic.go:356] "Generic (PLEG): container finished" podID="963fe765-b48e-484a-9904-567374393136" containerID="8ed2be328c57511ce5d858835f042bf11d1338a6cd3c346fe6e00360be48fee3" exitCode=0
Jul 21 13:19:59.915420 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:19:59.915409    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-04ee34-sbjsx" event={"ID":"963fe765-b48e-484a-9904-567374393136","Type":"ContainerDied","Data":"8ed2be328c57511ce5d858835f042bf11d1338a6cd3c346fe6e00360be48fee3"}
Jul 21 13:20:01.057216 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:20:01.057194    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-04ee34-sbjsx"
Jul 21 13:20:01.209656 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:20:01.209581    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/963fe765-b48e-484a-9904-567374393136-maas-api-service-ca\") pod \"963fe765-b48e-484a-9904-567374393136\" (UID: \"963fe765-b48e-484a-9904-567374393136\") "
Jul 21 13:20:01.209965 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:20:01.209935    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/963fe765-b48e-484a-9904-567374393136-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "963fe765-b48e-484a-9904-567374393136" (UID: "963fe765-b48e-484a-9904-567374393136"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 13:20:01.310476 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:20:01.310453    2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/963fe765-b48e-484a-9904-567374393136-maas-api-service-ca\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Jul 21 13:20:01.926688 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:20:01.926669    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-04ee34-sbjsx"
Jul 21 13:20:01.926838 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:20:01.926662    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-04ee34-sbjsx" event={"ID":"963fe765-b48e-484a-9904-567374393136","Type":"ContainerDied","Data":"82b3ae178c94420eda83090b0a3e660f4b4f820ce39ad0f1949f62ad874609e9"}
Jul 21 13:20:01.926838 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:20:01.926792    2556 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="82b3ae178c94420eda83090b0a3e660f4b4f820ce39ad0f1949f62ad874609e9"
Jul 21 13:20:03.236439 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:20:03.236411    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-d921bc-openshift-default-84d4d4f9b5-bnvgs"
Jul 21 13:20:03.327743 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:20:03.327685    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f7fe0978-0f99-4e5d-bbf3-32e8cd0610f7-workload-certs\") pod \"f7fe0978-0f99-4e5d-bbf3-32e8cd0610f7\" (UID: \"f7fe0978-0f99-4e5d-bbf3-32e8cd0610f7\") "
Jul 21 13:20:03.327866 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:20:03.327748    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f7fe0978-0f99-4e5d-bbf3-32e8cd0610f7-istio-podinfo\") pod \"f7fe0978-0f99-4e5d-bbf3-32e8cd0610f7\" (UID: \"f7fe0978-0f99-4e5d-bbf3-32e8cd0610f7\") "
Jul 21 13:20:03.327866 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:20:03.327769    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7fe0978-0f99-4e5d-bbf3-32e8cd0610f7-credential-socket\") pod \"f7fe0978-0f99-4e5d-bbf3-32e8cd0610f7\" (UID: \"f7fe0978-0f99-4e5d-bbf3-32e8cd0610f7\") "
Jul 21 13:20:03.327866 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:20:03.327797    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zkp4w\" (UniqueName: \"kubernetes.io/projected/f7fe0978-0f99-4e5d-bbf3-32e8cd0610f7-kube-api-access-zkp4w\") pod \"f7fe0978-0f99-4e5d-bbf3-32e8cd0610f7\" (UID: \"f7fe0978-0f99-4e5d-bbf3-32e8cd0610f7\") "
Jul 21 13:20:03.327866 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:20:03.327815    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f7fe0978-0f99-4e5d-bbf3-32e8cd0610f7-istio-token\") pod \"f7fe0978-0f99-4e5d-bbf3-32e8cd0610f7\" (UID: \"f7fe0978-0f99-4e5d-bbf3-32e8cd0610f7\") "
Jul 21 13:20:03.327866 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:20:03.327835    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7fe0978-0f99-4e5d-bbf3-32e8cd0610f7-workload-socket\") pod \"f7fe0978-0f99-4e5d-bbf3-32e8cd0610f7\" (UID: \"f7fe0978-0f99-4e5d-bbf3-32e8cd0610f7\") "
Jul 21 13:20:03.328169 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:20:03.327876    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f7fe0978-0f99-4e5d-bbf3-32e8cd0610f7-istio-envoy\") pod \"f7fe0978-0f99-4e5d-bbf3-32e8cd0610f7\" (UID: \"f7fe0978-0f99-4e5d-bbf3-32e8cd0610f7\") "
Jul 21 13:20:03.328169 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:20:03.327900    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f7fe0978-0f99-4e5d-bbf3-32e8cd0610f7-istiod-ca-cert\") pod \"f7fe0978-0f99-4e5d-bbf3-32e8cd0610f7\" (UID: \"f7fe0978-0f99-4e5d-bbf3-32e8cd0610f7\") "
Jul 21 13:20:03.328169 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:20:03.327945    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f7fe0978-0f99-4e5d-bbf3-32e8cd0610f7-istio-data\") pod \"f7fe0978-0f99-4e5d-bbf3-32e8cd0610f7\" (UID: \"f7fe0978-0f99-4e5d-bbf3-32e8cd0610f7\") "
Jul 21 13:20:03.328169 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:20:03.327972    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7fe0978-0f99-4e5d-bbf3-32e8cd0610f7-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f7fe0978-0f99-4e5d-bbf3-32e8cd0610f7" (UID: "f7fe0978-0f99-4e5d-bbf3-32e8cd0610f7"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 13:20:03.328169 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:20:03.328115    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7fe0978-0f99-4e5d-bbf3-32e8cd0610f7-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f7fe0978-0f99-4e5d-bbf3-32e8cd0610f7" (UID: "f7fe0978-0f99-4e5d-bbf3-32e8cd0610f7"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 13:20:03.328424 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:20:03.328355    2556 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7fe0978-0f99-4e5d-bbf3-32e8cd0610f7-credential-socket\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Jul 21 13:20:03.328424 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:20:03.328378    2556 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f7fe0978-0f99-4e5d-bbf3-32e8cd0610f7-workload-certs\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Jul 21 13:20:03.328819 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:20:03.328796    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7fe0978-0f99-4e5d-bbf3-32e8cd0610f7-istio-data" (OuterVolumeSpecName: "istio-data") pod "f7fe0978-0f99-4e5d-bbf3-32e8cd0610f7" (UID: "f7fe0978-0f99-4e5d-bbf3-32e8cd0610f7"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 13:20:03.328883 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:20:03.328790    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7fe0978-0f99-4e5d-bbf3-32e8cd0610f7-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f7fe0978-0f99-4e5d-bbf3-32e8cd0610f7" (UID: "f7fe0978-0f99-4e5d-bbf3-32e8cd0610f7"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 13:20:03.328917 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:20:03.328879    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f7fe0978-0f99-4e5d-bbf3-32e8cd0610f7-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f7fe0978-0f99-4e5d-bbf3-32e8cd0610f7" (UID: "f7fe0978-0f99-4e5d-bbf3-32e8cd0610f7"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 13:20:03.330780 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:20:03.330757    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f7fe0978-0f99-4e5d-bbf3-32e8cd0610f7-kube-api-access-zkp4w" (OuterVolumeSpecName: "kube-api-access-zkp4w") pod "f7fe0978-0f99-4e5d-bbf3-32e8cd0610f7" (UID: "f7fe0978-0f99-4e5d-bbf3-32e8cd0610f7"). InnerVolumeSpecName "kube-api-access-zkp4w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 13:20:03.330780 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:20:03.330758    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7fe0978-0f99-4e5d-bbf3-32e8cd0610f7-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f7fe0978-0f99-4e5d-bbf3-32e8cd0610f7" (UID: "f7fe0978-0f99-4e5d-bbf3-32e8cd0610f7"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 13:20:03.330943 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:20:03.330778    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f7fe0978-0f99-4e5d-bbf3-32e8cd0610f7-istio-token" (OuterVolumeSpecName: "istio-token") pod "f7fe0978-0f99-4e5d-bbf3-32e8cd0610f7" (UID: "f7fe0978-0f99-4e5d-bbf3-32e8cd0610f7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 13:20:03.330943 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:20:03.330807    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f7fe0978-0f99-4e5d-bbf3-32e8cd0610f7-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f7fe0978-0f99-4e5d-bbf3-32e8cd0610f7" (UID: "f7fe0978-0f99-4e5d-bbf3-32e8cd0610f7"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 21 13:20:03.428895 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:20:03.428872    2556 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f7fe0978-0f99-4e5d-bbf3-32e8cd0610f7-istio-data\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Jul 21 13:20:03.428895 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:20:03.428894    2556 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f7fe0978-0f99-4e5d-bbf3-32e8cd0610f7-istio-podinfo\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Jul 21 13:20:03.429060 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:20:03.428908    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zkp4w\" (UniqueName: \"kubernetes.io/projected/f7fe0978-0f99-4e5d-bbf3-32e8cd0610f7-kube-api-access-zkp4w\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Jul 21 13:20:03.429060 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:20:03.428920    2556 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f7fe0978-0f99-4e5d-bbf3-32e8cd0610f7-istio-token\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Jul 21 13:20:03.429060 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:20:03.428933    2556 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7fe0978-0f99-4e5d-bbf3-32e8cd0610f7-workload-socket\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Jul 21 13:20:03.429060 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:20:03.428945    2556 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f7fe0978-0f99-4e5d-bbf3-32e8cd0610f7-istio-envoy\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Jul 21 13:20:03.429060 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:20:03.428958    2556 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f7fe0978-0f99-4e5d-bbf3-32e8cd0610f7-istiod-ca-cert\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Jul 21 13:20:03.939162 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:20:03.939136    2556 generic.go:356] "Generic (PLEG): container finished" podID="f7fe0978-0f99-4e5d-bbf3-32e8cd0610f7" containerID="3f1b152bb3124562b7d500de742e28b40280d9726ab1ca6227f196e06854e0de" exitCode=0
Jul 21 13:20:03.939268 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:20:03.939183    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-d921bc-openshift-default-84d4d4f9b5-bnvgs"
Jul 21 13:20:03.939268 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:20:03.939192    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-d921bc-openshift-default-84d4d4f9b5-bnvgs" event={"ID":"f7fe0978-0f99-4e5d-bbf3-32e8cd0610f7","Type":"ContainerDied","Data":"3f1b152bb3124562b7d500de742e28b40280d9726ab1ca6227f196e06854e0de"}
Jul 21 13:20:03.939268 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:20:03.939233    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-d921bc-openshift-default-84d4d4f9b5-bnvgs" event={"ID":"f7fe0978-0f99-4e5d-bbf3-32e8cd0610f7","Type":"ContainerDied","Data":"2362804ca25c884820859c26b8446b427a35a07b3b890ac7ff83520911cfc9ab"}
Jul 21 13:20:03.939268 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:20:03.939254    2556 scope.go:117] "RemoveContainer" containerID="3f1b152bb3124562b7d500de742e28b40280d9726ab1ca6227f196e06854e0de"
Jul 21 13:20:03.948708 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:20:03.948692    2556 scope.go:117] "RemoveContainer" containerID="08f201cb6d5b625cafb3760549cc6a1b11d38dd4167a7ad07b00e08fcb7ba0e4"
Jul 21 13:20:03.956722 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:20:03.956706    2556 scope.go:117] "RemoveContainer" containerID="3f1b152bb3124562b7d500de742e28b40280d9726ab1ca6227f196e06854e0de"
Jul 21 13:20:03.956977 ip-10-0-130-2 kubenswrapper[2556]: E0721 13:20:03.956958    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3f1b152bb3124562b7d500de742e28b40280d9726ab1ca6227f196e06854e0de\": container with ID starting with 3f1b152bb3124562b7d500de742e28b40280d9726ab1ca6227f196e06854e0de not found: ID does not exist" containerID="3f1b152bb3124562b7d500de742e28b40280d9726ab1ca6227f196e06854e0de"
Jul 21 13:20:03.957067 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:20:03.956983    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3f1b152bb3124562b7d500de742e28b40280d9726ab1ca6227f196e06854e0de"} err="failed to get container status \"3f1b152bb3124562b7d500de742e28b40280d9726ab1ca6227f196e06854e0de\": rpc error: code = NotFound desc = could not find container \"3f1b152bb3124562b7d500de742e28b40280d9726ab1ca6227f196e06854e0de\": container with ID starting with 3f1b152bb3124562b7d500de742e28b40280d9726ab1ca6227f196e06854e0de not found: ID does not exist"
Jul 21 13:20:03.957067 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:20:03.956999    2556 scope.go:117] "RemoveContainer" containerID="08f201cb6d5b625cafb3760549cc6a1b11d38dd4167a7ad07b00e08fcb7ba0e4"
Jul 21 13:20:03.957259 ip-10-0-130-2 kubenswrapper[2556]: E0721 13:20:03.957243    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"08f201cb6d5b625cafb3760549cc6a1b11d38dd4167a7ad07b00e08fcb7ba0e4\": container with ID starting with 08f201cb6d5b625cafb3760549cc6a1b11d38dd4167a7ad07b00e08fcb7ba0e4 not found: ID does not exist" containerID="08f201cb6d5b625cafb3760549cc6a1b11d38dd4167a7ad07b00e08fcb7ba0e4"
Jul 21 13:20:03.957299 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:20:03.957266    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"08f201cb6d5b625cafb3760549cc6a1b11d38dd4167a7ad07b00e08fcb7ba0e4"} err="failed to get container status \"08f201cb6d5b625cafb3760549cc6a1b11d38dd4167a7ad07b00e08fcb7ba0e4\": rpc error: code = NotFound desc = could not find container \"08f201cb6d5b625cafb3760549cc6a1b11d38dd4167a7ad07b00e08fcb7ba0e4\": container with ID starting with 08f201cb6d5b625cafb3760549cc6a1b11d38dd4167a7ad07b00e08fcb7ba0e4 not found: ID does not exist"
Jul 21 13:20:03.962232 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:20:03.962212    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-d921bc-openshift-default-84d4d4f9b5-bnvgs"]
Jul 21 13:20:03.967939 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:20:03.967919    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-d921bc-openshift-default-84d4d4f9b5-bnvgs"]
Jul 21 13:20:05.194521 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:20:05.194485    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f7fe0978-0f99-4e5d-bbf3-32e8cd0610f7" path="/var/lib/kubelet/pods/f7fe0978-0f99-4e5d-bbf3-32e8cd0610f7/volumes"
Jul 21 13:20:18.490969 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:20:18.490864    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-04ee34-sbjsx"]
Jul 21 13:20:18.493155 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:20:18.493133    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-04ee34-sbjsx"]
Jul 21 13:20:19.200664 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:20:19.200626    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="963fe765-b48e-484a-9904-567374393136" path="/var/lib/kubelet/pods/963fe765-b48e-484a-9904-567374393136/volumes"
Jul 21 13:20:26.986303 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:20:26.986267    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-b3a013-openshift-default-5648c9c987-nzzwb"]
Jul 21 13:20:26.986805 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:20:26.986746    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7fe0978-0f99-4e5d-bbf3-32e8cd0610f7" containerName="istio-proxy"
Jul 21 13:20:26.986805 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:20:26.986758    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7fe0978-0f99-4e5d-bbf3-32e8cd0610f7" containerName="istio-proxy"
Jul 21 13:20:26.986805 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:20:26.986774    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="963fe765-b48e-484a-9904-567374393136" containerName="revoke-keys"
Jul 21 13:20:26.986805 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:20:26.986779    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="963fe765-b48e-484a-9904-567374393136" containerName="revoke-keys"
Jul 21 13:20:26.986938 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:20:26.986860    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="f7fe0978-0f99-4e5d-bbf3-32e8cd0610f7" containerName="istio-proxy"
Jul 21 13:20:26.986938 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:20:26.986869    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="f7fe0978-0f99-4e5d-bbf3-32e8cd0610f7" containerName="istio-proxy"
Jul 21 13:20:26.986938 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:20:26.986876    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="963fe765-b48e-484a-9904-567374393136" containerName="revoke-keys"
Jul 21 13:20:26.990116 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:20:26.990095    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-b3a013-openshift-default-5648c9c987-nzzwb"
Jul 21 13:20:26.993416 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:20:26.993389    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-b3a013-openshift-default-dockercfg-gcx7t\""
Jul 21 13:20:27.009866 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:20:27.009837    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-b3a013-openshift-default-5648c9c987-nzzwb"]
Jul 21 13:20:27.020443 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:20:27.020418    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b0034705-9d87-44f2-b191-aabfaabfeb6e-istio-podinfo\") pod \"e2e-shared-a-b3a013-openshift-default-5648c9c987-nzzwb\" (UID: \"b0034705-9d87-44f2-b191-aabfaabfeb6e\") " pod="openshift-ingress/e2e-shared-a-b3a013-openshift-default-5648c9c987-nzzwb"
Jul 21 13:20:27.020575 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:20:27.020453    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0034705-9d87-44f2-b191-aabfaabfeb6e-credential-socket\") pod \"e2e-shared-a-b3a013-openshift-default-5648c9c987-nzzwb\" (UID: \"b0034705-9d87-44f2-b191-aabfaabfeb6e\") " pod="openshift-ingress/e2e-shared-a-b3a013-openshift-default-5648c9c987-nzzwb"
Jul 21 13:20:27.020575 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:20:27.020479    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b0034705-9d87-44f2-b191-aabfaabfeb6e-istio-token\") pod \"e2e-shared-a-b3a013-openshift-default-5648c9c987-nzzwb\" (UID: \"b0034705-9d87-44f2-b191-aabfaabfeb6e\") " pod="openshift-ingress/e2e-shared-a-b3a013-openshift-default-5648c9c987-nzzwb"
Jul 21 13:20:27.020575 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:20:27.020526    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0034705-9d87-44f2-b191-aabfaabfeb6e-workload-socket\") pod \"e2e-shared-a-b3a013-openshift-default-5648c9c987-nzzwb\" (UID: \"b0034705-9d87-44f2-b191-aabfaabfeb6e\") " pod="openshift-ingress/e2e-shared-a-b3a013-openshift-default-5648c9c987-nzzwb"
Jul 21 13:20:27.020575 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:20:27.020561    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b0034705-9d87-44f2-b191-aabfaabfeb6e-istio-data\") pod \"e2e-shared-a-b3a013-openshift-default-5648c9c987-nzzwb\" (UID: \"b0034705-9d87-44f2-b191-aabfaabfeb6e\") " pod="openshift-ingress/e2e-shared-a-b3a013-openshift-default-5648c9c987-nzzwb"
Jul 21 13:20:27.020803 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:20:27.020578    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9894l\" (UniqueName: \"kubernetes.io/projected/b0034705-9d87-44f2-b191-aabfaabfeb6e-kube-api-access-9894l\") pod \"e2e-shared-a-b3a013-openshift-default-5648c9c987-nzzwb\" (UID: \"b0034705-9d87-44f2-b191-aabfaabfeb6e\") " pod="openshift-ingress/e2e-shared-a-b3a013-openshift-default-5648c9c987-nzzwb"
Jul 21 13:20:27.020803 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:20:27.020618    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b0034705-9d87-44f2-b191-aabfaabfeb6e-istiod-ca-cert\") pod \"e2e-shared-a-b3a013-openshift-default-5648c9c987-nzzwb\" (UID: \"b0034705-9d87-44f2-b191-aabfaabfeb6e\") " pod="openshift-ingress/e2e-shared-a-b3a013-openshift-default-5648c9c987-nzzwb"
Jul 21 13:20:27.020803 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:20:27.020675    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b0034705-9d87-44f2-b191-aabfaabfeb6e-workload-certs\") pod \"e2e-shared-a-b3a013-openshift-default-5648c9c987-nzzwb\" (UID: \"b0034705-9d87-44f2-b191-aabfaabfeb6e\") " pod="openshift-ingress/e2e-shared-a-b3a013-openshift-default-5648c9c987-nzzwb"
Jul 21 13:20:27.020803 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:20:27.020692    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b0034705-9d87-44f2-b191-aabfaabfeb6e-istio-envoy\") pod \"e2e-shared-a-b3a013-openshift-default-5648c9c987-nzzwb\" (UID: \"b0034705-9d87-44f2-b191-aabfaabfeb6e\") " pod="openshift-ingress/e2e-shared-a-b3a013-openshift-default-5648c9c987-nzzwb"
Jul 21 13:20:27.121929 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:20:27.121893    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b0034705-9d87-44f2-b191-aabfaabfeb6e-istio-podinfo\") pod \"e2e-shared-a-b3a013-openshift-default-5648c9c987-nzzwb\" (UID: \"b0034705-9d87-44f2-b191-aabfaabfeb6e\") " pod="openshift-ingress/e2e-shared-a-b3a013-openshift-default-5648c9c987-nzzwb"
Jul 21 13:20:27.121929 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:20:27.121935    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0034705-9d87-44f2-b191-aabfaabfeb6e-credential-socket\") pod \"e2e-shared-a-b3a013-openshift-default-5648c9c987-nzzwb\" (UID: \"b0034705-9d87-44f2-b191-aabfaabfeb6e\") " pod="openshift-ingress/e2e-shared-a-b3a013-openshift-default-5648c9c987-nzzwb"
Jul 21 13:20:27.122166 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:20:27.122076    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b0034705-9d87-44f2-b191-aabfaabfeb6e-istio-token\") pod \"e2e-shared-a-b3a013-openshift-default-5648c9c987-nzzwb\" (UID: \"b0034705-9d87-44f2-b191-aabfaabfeb6e\") " pod="openshift-ingress/e2e-shared-a-b3a013-openshift-default-5648c9c987-nzzwb"
Jul 21 13:20:27.122166 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:20:27.122111    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0034705-9d87-44f2-b191-aabfaabfeb6e-workload-socket\") pod \"e2e-shared-a-b3a013-openshift-default-5648c9c987-nzzwb\" (UID: \"b0034705-9d87-44f2-b191-aabfaabfeb6e\") " pod="openshift-ingress/e2e-shared-a-b3a013-openshift-default-5648c9c987-nzzwb"
Jul 21 13:20:27.122271 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:20:27.122245    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b0034705-9d87-44f2-b191-aabfaabfeb6e-istio-data\") pod \"e2e-shared-a-b3a013-openshift-default-5648c9c987-nzzwb\" (UID: \"b0034705-9d87-44f2-b191-aabfaabfeb6e\") " pod="openshift-ingress/e2e-shared-a-b3a013-openshift-default-5648c9c987-nzzwb"
Jul 21 13:20:27.122320 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:20:27.122291    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9894l\" (UniqueName: \"kubernetes.io/projected/b0034705-9d87-44f2-b191-aabfaabfeb6e-kube-api-access-9894l\") pod \"e2e-shared-a-b3a013-openshift-default-5648c9c987-nzzwb\" (UID: \"b0034705-9d87-44f2-b191-aabfaabfeb6e\") " pod="openshift-ingress/e2e-shared-a-b3a013-openshift-default-5648c9c987-nzzwb"
Jul 21 13:20:27.122363 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:20:27.122336    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0034705-9d87-44f2-b191-aabfaabfeb6e-credential-socket\") pod \"e2e-shared-a-b3a013-openshift-default-5648c9c987-nzzwb\" (UID: \"b0034705-9d87-44f2-b191-aabfaabfeb6e\") " pod="openshift-ingress/e2e-shared-a-b3a013-openshift-default-5648c9c987-nzzwb"
Jul 21 13:20:27.122410 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:20:27.122359    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b0034705-9d87-44f2-b191-aabfaabfeb6e-istiod-ca-cert\") pod \"e2e-shared-a-b3a013-openshift-default-5648c9c987-nzzwb\" (UID: \"b0034705-9d87-44f2-b191-aabfaabfeb6e\") " pod="openshift-ingress/e2e-shared-a-b3a013-openshift-default-5648c9c987-nzzwb"
Jul 21 13:20:27.122410 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:20:27.122392    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0034705-9d87-44f2-b191-aabfaabfeb6e-workload-socket\") pod \"e2e-shared-a-b3a013-openshift-default-5648c9c987-nzzwb\" (UID: \"b0034705-9d87-44f2-b191-aabfaabfeb6e\") " pod="openshift-ingress/e2e-shared-a-b3a013-openshift-default-5648c9c987-nzzwb"
Jul 21 13:20:27.122520 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:20:27.122426    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b0034705-9d87-44f2-b191-aabfaabfeb6e-workload-certs\") pod \"e2e-shared-a-b3a013-openshift-default-5648c9c987-nzzwb\" (UID: \"b0034705-9d87-44f2-b191-aabfaabfeb6e\") " pod="openshift-ingress/e2e-shared-a-b3a013-openshift-default-5648c9c987-nzzwb"
Jul 21 13:20:27.122520 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:20:27.122453    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b0034705-9d87-44f2-b191-aabfaabfeb6e-istio-envoy\") pod \"e2e-shared-a-b3a013-openshift-default-5648c9c987-nzzwb\" (UID: \"b0034705-9d87-44f2-b191-aabfaabfeb6e\") " pod="openshift-ingress/e2e-shared-a-b3a013-openshift-default-5648c9c987-nzzwb"
Jul 21 13:20:27.122618 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:20:27.122523    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b0034705-9d87-44f2-b191-aabfaabfeb6e-istio-data\") pod \"e2e-shared-a-b3a013-openshift-default-5648c9c987-nzzwb\" (UID: \"b0034705-9d87-44f2-b191-aabfaabfeb6e\") " pod="openshift-ingress/e2e-shared-a-b3a013-openshift-default-5648c9c987-nzzwb"
Jul 21 13:20:27.122837 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:20:27.122775    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b0034705-9d87-44f2-b191-aabfaabfeb6e-workload-certs\") pod \"e2e-shared-a-b3a013-openshift-default-5648c9c987-nzzwb\" (UID: \"b0034705-9d87-44f2-b191-aabfaabfeb6e\") " pod="openshift-ingress/e2e-shared-a-b3a013-openshift-default-5648c9c987-nzzwb"
Jul 21 13:20:27.122976 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:20:27.122908    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b0034705-9d87-44f2-b191-aabfaabfeb6e-istiod-ca-cert\") pod \"e2e-shared-a-b3a013-openshift-default-5648c9c987-nzzwb\" (UID: \"b0034705-9d87-44f2-b191-aabfaabfeb6e\") " pod="openshift-ingress/e2e-shared-a-b3a013-openshift-default-5648c9c987-nzzwb"
Jul 21 13:20:27.124652 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:20:27.124630    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b0034705-9d87-44f2-b191-aabfaabfeb6e-istio-podinfo\") pod \"e2e-shared-a-b3a013-openshift-default-5648c9c987-nzzwb\" (UID: \"b0034705-9d87-44f2-b191-aabfaabfeb6e\") " pod="openshift-ingress/e2e-shared-a-b3a013-openshift-default-5648c9c987-nzzwb"
Jul 21 13:20:27.124758 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:20:27.124702    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b0034705-9d87-44f2-b191-aabfaabfeb6e-istio-envoy\") pod \"e2e-shared-a-b3a013-openshift-default-5648c9c987-nzzwb\" (UID: \"b0034705-9d87-44f2-b191-aabfaabfeb6e\") " pod="openshift-ingress/e2e-shared-a-b3a013-openshift-default-5648c9c987-nzzwb"
Jul 21 13:20:27.136355 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:20:27.136331    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b0034705-9d87-44f2-b191-aabfaabfeb6e-istio-token\") pod \"e2e-shared-a-b3a013-openshift-default-5648c9c987-nzzwb\" (UID: \"b0034705-9d87-44f2-b191-aabfaabfeb6e\") " pod="openshift-ingress/e2e-shared-a-b3a013-openshift-default-5648c9c987-nzzwb"
Jul 21 13:20:27.136802 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:20:27.136781    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9894l\" (UniqueName: \"kubernetes.io/projected/b0034705-9d87-44f2-b191-aabfaabfeb6e-kube-api-access-9894l\") pod \"e2e-shared-a-b3a013-openshift-default-5648c9c987-nzzwb\" (UID: \"b0034705-9d87-44f2-b191-aabfaabfeb6e\") " pod="openshift-ingress/e2e-shared-a-b3a013-openshift-default-5648c9c987-nzzwb"
Jul 21 13:20:27.301448 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:20:27.301367    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-b3a013-openshift-default-5648c9c987-nzzwb"
Jul 21 13:20:27.448204 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:20:27.448173    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-b3a013-openshift-default-5648c9c987-nzzwb"]
Jul 21 13:20:27.449898 ip-10-0-130-2 kubenswrapper[2556]: W0721 13:20:27.449873    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb0034705_9d87_44f2_b191_aabfaabfeb6e.slice/crio-ac8995b5bdd969acfab3130141e715a7a51f0aaff5705c4b1b4cc245668d3405 WatchSource:0}: Error finding container ac8995b5bdd969acfab3130141e715a7a51f0aaff5705c4b1b4cc245668d3405: Status 404 returned error can't find the container with id ac8995b5bdd969acfab3130141e715a7a51f0aaff5705c4b1b4cc245668d3405
Jul 21 13:20:27.452096 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:20:27.452066    2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 21 13:20:27.452167 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:20:27.452131    2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 21 13:20:27.452204 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:20:27.452161    2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 21 13:20:28.065872 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:20:28.065834    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-b3a013-openshift-default-5648c9c987-nzzwb" event={"ID":"b0034705-9d87-44f2-b191-aabfaabfeb6e","Type":"ContainerStarted","Data":"0790c499256d1e026d0294b67a8c773d5abbcfd550b32d3a4dcb98799c469a64"}
Jul 21 13:20:28.066365 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:20:28.065889    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-b3a013-openshift-default-5648c9c987-nzzwb" event={"ID":"b0034705-9d87-44f2-b191-aabfaabfeb6e","Type":"ContainerStarted","Data":"ac8995b5bdd969acfab3130141e715a7a51f0aaff5705c4b1b4cc245668d3405"}
Jul 21 13:20:28.094050 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:20:28.093970    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-b3a013-openshift-default-5648c9c987-nzzwb" podStartSLOduration=2.093949981 podStartE2EDuration="2.093949981s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 13:20:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:20:28.091664991 +0000 UTC m=+2441.395670094" watchObservedRunningTime="2026-07-21 13:20:28.093949981 +0000 UTC m=+2441.397955072"
Jul 21 13:20:28.301697 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:20:28.301661    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-b3a013-openshift-default-5648c9c987-nzzwb"
Jul 21 13:20:28.307156 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:20:28.307131    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-b3a013-openshift-default-5648c9c987-nzzwb"
Jul 21 13:20:29.071929 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:20:29.071897    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-b3a013-openshift-default-5648c9c987-nzzwb"
Jul 21 13:20:29.072993 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:20:29.072970    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-b3a013-openshift-default-5648c9c987-nzzwb"
Jul 21 13:20:44.072936 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:20:44.072896    2556 patch_prober.go:28] interesting pod/e2e-shared-a-b3a013-openshift-default-5648c9c987-nzzwb container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body=
Jul 21 13:20:44.073375 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:20:44.072960    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-b3a013-openshift-default-5648c9c987-nzzwb" podUID="b0034705-9d87-44f2-b191-aabfaabfeb6e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused"
Jul 21 13:20:44.161824 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:20:44.161780    2556 generic.go:356] "Generic (PLEG): container finished" podID="b0034705-9d87-44f2-b191-aabfaabfeb6e" containerID="0790c499256d1e026d0294b67a8c773d5abbcfd550b32d3a4dcb98799c469a64" exitCode=0
Jul 21 13:20:44.161992 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:20:44.161853    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-b3a013-openshift-default-5648c9c987-nzzwb" event={"ID":"b0034705-9d87-44f2-b191-aabfaabfeb6e","Type":"ContainerDied","Data":"0790c499256d1e026d0294b67a8c773d5abbcfd550b32d3a4dcb98799c469a64"}
Jul 21 13:20:44.162346 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:20:44.162328    2556 scope.go:117] "RemoveContainer" containerID="0790c499256d1e026d0294b67a8c773d5abbcfd550b32d3a4dcb98799c469a64"
Jul 21 13:20:44.163478 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:20:44.163447    2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 21 13:20:44.163588 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:20:44.163545    2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 21 13:20:44.163651 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:20:44.163596    2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 21 13:20:45.168868 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:20:45.168817    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-b3a013-openshift-default-5648c9c987-nzzwb" event={"ID":"b0034705-9d87-44f2-b191-aabfaabfeb6e","Type":"ContainerStarted","Data":"5b3e09930c2ac4a297d0e4c63813997c546a76c6a896ddc48a549528af8087a2"}
Jul 21 13:20:45.302395 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:20:45.302353    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-b3a013-openshift-default-5648c9c987-nzzwb"
Jul 21 13:20:45.302671 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:20:45.302642    2556 patch_prober.go:28] interesting pod/e2e-shared-a-b3a013-openshift-default-5648c9c987-nzzwb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body=
Jul 21 13:20:45.302755 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:20:45.302720    2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-b3a013-openshift-default-5648c9c987-nzzwb" podUID="b0034705-9d87-44f2-b191-aabfaabfeb6e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused"
Jul 21 13:20:46.301910 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:20:46.301870    2556 patch_prober.go:28] interesting pod/e2e-shared-a-b3a013-openshift-default-5648c9c987-nzzwb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body=
Jul 21 13:20:46.302425 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:20:46.301953    2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-b3a013-openshift-default-5648c9c987-nzzwb" podUID="b0034705-9d87-44f2-b191-aabfaabfeb6e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused"
Jul 21 13:20:47.302236 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:20:47.302180    2556 patch_prober.go:28] interesting pod/e2e-shared-a-b3a013-openshift-default-5648c9c987-nzzwb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body=
Jul 21 13:20:47.302743 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:20:47.302264    2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-b3a013-openshift-default-5648c9c987-nzzwb" podUID="b0034705-9d87-44f2-b191-aabfaabfeb6e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused"
Jul 21 13:20:47.511008 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:20:47.510978    2556 scope.go:117] "RemoveContainer" containerID="6be8aff0640f245550a9603a9e6fba9a8237e3878dc12824e3d3b4d3fa92ddc9"
Jul 21 13:20:48.302312 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:20:48.302278    2556 patch_prober.go:28] interesting pod/e2e-shared-a-b3a013-openshift-default-5648c9c987-nzzwb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body=
Jul 21 13:20:48.302894 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:20:48.302864    2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-b3a013-openshift-default-5648c9c987-nzzwb" podUID="b0034705-9d87-44f2-b191-aabfaabfeb6e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused"
Jul 21 13:20:49.305913 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:20:49.305885    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-b3a013-openshift-default-5648c9c987-nzzwb"
Jul 21 13:20:49.306337 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:20:49.306272    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-b3a013-openshift-default-5648c9c987-nzzwb"
Jul 21 13:20:49.306914 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:20:49.306893    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-b3a013-openshift-default-5648c9c987-nzzwb"
Jul 21 13:20:59.865096 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:20:59.865061    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-b3a013/auth-test-model-b3a013-kserve-79d7f98b99-qfrjp"]
Jul 21 13:20:59.865868 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:20:59.865490    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7fe0978-0f99-4e5d-bbf3-32e8cd0610f7" containerName="istio-proxy"
Jul 21 13:20:59.865868 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:20:59.865500    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7fe0978-0f99-4e5d-bbf3-32e8cd0610f7" containerName="istio-proxy"
Jul 21 13:20:59.868730 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:20:59.868712    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-b3a013/auth-test-model-b3a013-kserve-79d7f98b99-qfrjp"
Jul 21 13:20:59.871409 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:20:59.871388    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-b3a013\"/\"kube-root-ca.crt\""
Jul 21 13:20:59.871920 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:20:59.871895    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-b3a013\"/\"openshift-service-ca.crt\""
Jul 21 13:20:59.871920 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:20:59.871910    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-b3a013\"/\"default-dockercfg-bt7lb\""
Jul 21 13:20:59.872157 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:20:59.871996    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-b3a013\"/\"auth-test-model-b3a013-kserve-self-signed-certs\""
Jul 21 13:20:59.878590 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:20:59.878570    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-b3a013/auth-test-model-b3a013-kserve-79d7f98b99-qfrjp"]
Jul 21 13:21:00.040500 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:21:00.040471    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f004a996-efff-468e-8501-e464d98ee25d-tmp-dir\") pod \"auth-test-model-b3a013-kserve-79d7f98b99-qfrjp\" (UID: \"f004a996-efff-468e-8501-e464d98ee25d\") " pod="ai-tenant-e2e-shared-a-b3a013/auth-test-model-b3a013-kserve-79d7f98b99-qfrjp"
Jul 21 13:21:00.040676 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:21:00.040518    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f004a996-efff-468e-8501-e464d98ee25d-tls-certs\") pod \"auth-test-model-b3a013-kserve-79d7f98b99-qfrjp\" (UID: \"f004a996-efff-468e-8501-e464d98ee25d\") " pod="ai-tenant-e2e-shared-a-b3a013/auth-test-model-b3a013-kserve-79d7f98b99-qfrjp"
Jul 21 13:21:00.040676 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:21:00.040594    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f004a996-efff-468e-8501-e464d98ee25d-model-cache\") pod \"auth-test-model-b3a013-kserve-79d7f98b99-qfrjp\" (UID: \"f004a996-efff-468e-8501-e464d98ee25d\") " pod="ai-tenant-e2e-shared-a-b3a013/auth-test-model-b3a013-kserve-79d7f98b99-qfrjp"
Jul 21 13:21:00.040676 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:21:00.040658    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f004a996-efff-468e-8501-e464d98ee25d-home\") pod \"auth-test-model-b3a013-kserve-79d7f98b99-qfrjp\" (UID: \"f004a996-efff-468e-8501-e464d98ee25d\") " pod="ai-tenant-e2e-shared-a-b3a013/auth-test-model-b3a013-kserve-79d7f98b99-qfrjp"
Jul 21 13:21:00.040814 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:21:00.040711    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f004a996-efff-468e-8501-e464d98ee25d-dshm\") pod \"auth-test-model-b3a013-kserve-79d7f98b99-qfrjp\" (UID: \"f004a996-efff-468e-8501-e464d98ee25d\") " pod="ai-tenant-e2e-shared-a-b3a013/auth-test-model-b3a013-kserve-79d7f98b99-qfrjp"
Jul 21 13:21:00.040814 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:21:00.040737    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h5flq\" (UniqueName: \"kubernetes.io/projected/f004a996-efff-468e-8501-e464d98ee25d-kube-api-access-h5flq\") pod \"auth-test-model-b3a013-kserve-79d7f98b99-qfrjp\" (UID: \"f004a996-efff-468e-8501-e464d98ee25d\") " pod="ai-tenant-e2e-shared-a-b3a013/auth-test-model-b3a013-kserve-79d7f98b99-qfrjp"
Jul 21 13:21:00.142456 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:21:00.142354    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f004a996-efff-468e-8501-e464d98ee25d-tmp-dir\") pod \"auth-test-model-b3a013-kserve-79d7f98b99-qfrjp\" (UID: \"f004a996-efff-468e-8501-e464d98ee25d\") " pod="ai-tenant-e2e-shared-a-b3a013/auth-test-model-b3a013-kserve-79d7f98b99-qfrjp"
Jul 21 13:21:00.142456 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:21:00.142407    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f004a996-efff-468e-8501-e464d98ee25d-tls-certs\") pod \"auth-test-model-b3a013-kserve-79d7f98b99-qfrjp\" (UID: \"f004a996-efff-468e-8501-e464d98ee25d\") " pod="ai-tenant-e2e-shared-a-b3a013/auth-test-model-b3a013-kserve-79d7f98b99-qfrjp"
Jul 21 13:21:00.142456 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:21:00.142446    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f004a996-efff-468e-8501-e464d98ee25d-model-cache\") pod \"auth-test-model-b3a013-kserve-79d7f98b99-qfrjp\" (UID: \"f004a996-efff-468e-8501-e464d98ee25d\") " pod="ai-tenant-e2e-shared-a-b3a013/auth-test-model-b3a013-kserve-79d7f98b99-qfrjp"
Jul 21 13:21:00.142769 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:21:00.142483    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f004a996-efff-468e-8501-e464d98ee25d-home\") pod \"auth-test-model-b3a013-kserve-79d7f98b99-qfrjp\" (UID: \"f004a996-efff-468e-8501-e464d98ee25d\") " pod="ai-tenant-e2e-shared-a-b3a013/auth-test-model-b3a013-kserve-79d7f98b99-qfrjp"
Jul 21 13:21:00.142769 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:21:00.142598    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f004a996-efff-468e-8501-e464d98ee25d-dshm\") pod \"auth-test-model-b3a013-kserve-79d7f98b99-qfrjp\" (UID: \"f004a996-efff-468e-8501-e464d98ee25d\") " pod="ai-tenant-e2e-shared-a-b3a013/auth-test-model-b3a013-kserve-79d7f98b99-qfrjp"
Jul 21 13:21:00.142769 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:21:00.142659    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h5flq\" (UniqueName: \"kubernetes.io/projected/f004a996-efff-468e-8501-e464d98ee25d-kube-api-access-h5flq\") pod \"auth-test-model-b3a013-kserve-79d7f98b99-qfrjp\" (UID: \"f004a996-efff-468e-8501-e464d98ee25d\") " pod="ai-tenant-e2e-shared-a-b3a013/auth-test-model-b3a013-kserve-79d7f98b99-qfrjp"
Jul 21 13:21:00.142949 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:21:00.142800    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f004a996-efff-468e-8501-e464d98ee25d-model-cache\") pod \"auth-test-model-b3a013-kserve-79d7f98b99-qfrjp\" (UID: \"f004a996-efff-468e-8501-e464d98ee25d\") " pod="ai-tenant-e2e-shared-a-b3a013/auth-test-model-b3a013-kserve-79d7f98b99-qfrjp"
Jul 21 13:21:00.142949 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:21:00.142855    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f004a996-efff-468e-8501-e464d98ee25d-home\") pod \"auth-test-model-b3a013-kserve-79d7f98b99-qfrjp\" (UID: \"f004a996-efff-468e-8501-e464d98ee25d\") " pod="ai-tenant-e2e-shared-a-b3a013/auth-test-model-b3a013-kserve-79d7f98b99-qfrjp"
Jul 21 13:21:00.142949 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:21:00.142880    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f004a996-efff-468e-8501-e464d98ee25d-tmp-dir\") pod \"auth-test-model-b3a013-kserve-79d7f98b99-qfrjp\" (UID: \"f004a996-efff-468e-8501-e464d98ee25d\") " pod="ai-tenant-e2e-shared-a-b3a013/auth-test-model-b3a013-kserve-79d7f98b99-qfrjp"
Jul 21 13:21:00.145000 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:21:00.144969    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f004a996-efff-468e-8501-e464d98ee25d-dshm\") pod \"auth-test-model-b3a013-kserve-79d7f98b99-qfrjp\" (UID: \"f004a996-efff-468e-8501-e464d98ee25d\") " pod="ai-tenant-e2e-shared-a-b3a013/auth-test-model-b3a013-kserve-79d7f98b99-qfrjp"
Jul 21 13:21:00.145256 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:21:00.145240    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f004a996-efff-468e-8501-e464d98ee25d-tls-certs\") pod \"auth-test-model-b3a013-kserve-79d7f98b99-qfrjp\" (UID: \"f004a996-efff-468e-8501-e464d98ee25d\") " pod="ai-tenant-e2e-shared-a-b3a013/auth-test-model-b3a013-kserve-79d7f98b99-qfrjp"
Jul 21 13:21:00.152252 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:21:00.152232    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h5flq\" (UniqueName: \"kubernetes.io/projected/f004a996-efff-468e-8501-e464d98ee25d-kube-api-access-h5flq\") pod \"auth-test-model-b3a013-kserve-79d7f98b99-qfrjp\" (UID: \"f004a996-efff-468e-8501-e464d98ee25d\") " pod="ai-tenant-e2e-shared-a-b3a013/auth-test-model-b3a013-kserve-79d7f98b99-qfrjp"
Jul 21 13:21:00.180097 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:21:00.180072    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-b3a013/auth-test-model-b3a013-kserve-79d7f98b99-qfrjp"
Jul 21 13:21:00.321717 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:21:00.321691    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-b3a013/auth-test-model-b3a013-kserve-79d7f98b99-qfrjp"]
Jul 21 13:21:00.323998 ip-10-0-130-2 kubenswrapper[2556]: W0721 13:21:00.323970    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf004a996_efff_468e_8501_e464d98ee25d.slice/crio-36bd7913823494c156aa7c6367f9a5b813460ee6f86e68bed44d368dd9e4dedc WatchSource:0}: Error finding container 36bd7913823494c156aa7c6367f9a5b813460ee6f86e68bed44d368dd9e4dedc: Status 404 returned error can't find the container with id 36bd7913823494c156aa7c6367f9a5b813460ee6f86e68bed44d368dd9e4dedc
Jul 21 13:21:01.259446 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:21:01.259404    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-b3a013/auth-test-model-b3a013-kserve-79d7f98b99-qfrjp" event={"ID":"f004a996-efff-468e-8501-e464d98ee25d","Type":"ContainerStarted","Data":"d2f8dc0789f96f553f0f57a3c9924aaed8d9058dd74087ee64ac6854a0ea95f2"}
Jul 21 13:21:01.259896 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:21:01.259830    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-b3a013/auth-test-model-b3a013-kserve-79d7f98b99-qfrjp" event={"ID":"f004a996-efff-468e-8501-e464d98ee25d","Type":"ContainerStarted","Data":"36bd7913823494c156aa7c6367f9a5b813460ee6f86e68bed44d368dd9e4dedc"}
Jul 21 13:21:01.279639 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:21:01.279593    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-b3a013/auth-test-model-b3a013-kserve-79d7f98b99-qfrjp" podStartSLOduration=2.279580765 podStartE2EDuration="2.279580765s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 13:20:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:21:01.278115182 +0000 UTC m=+2474.582120282" watchObservedRunningTime="2026-07-21 13:21:01.279580765 +0000 UTC m=+2474.583585854"
Jul 21 13:21:10.181107 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:21:10.181072    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-b3a013/auth-test-model-b3a013-kserve-79d7f98b99-qfrjp"
Jul 21 13:21:10.181483 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:21:10.181125    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-b3a013/auth-test-model-b3a013-kserve-79d7f98b99-qfrjp"
Jul 21 13:21:10.193950 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:21:10.193925    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-b3a013/auth-test-model-b3a013-kserve-79d7f98b99-qfrjp"
Jul 21 13:21:10.317817 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:21:10.317789    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-b3a013/auth-test-model-b3a013-kserve-79d7f98b99-qfrjp"
Jul 21 13:23:28.133930 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:23:28.133845    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-b3a013/auth-test-model-b3a013-kserve-79d7f98b99-qfrjp"]
Jul 21 13:23:43.154704 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:23:43.154663    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-b3a013/auth-test-model-b3a013-kserve-79d7f98b99-qfrjp" podUID="f004a996-efff-468e-8501-e464d98ee25d" containerName="main" containerID="cri-o://d2f8dc0789f96f553f0f57a3c9924aaed8d9058dd74087ee64ac6854a0ea95f2" gracePeriod=45
Jul 21 13:23:43.415622 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:23:43.415560    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-b3a013/auth-test-model-b3a013-kserve-79d7f98b99-qfrjp"
Jul 21 13:23:43.493100 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:23:43.493054    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f004a996-efff-468e-8501-e464d98ee25d-dshm\") pod \"f004a996-efff-468e-8501-e464d98ee25d\" (UID: \"f004a996-efff-468e-8501-e464d98ee25d\") "
Jul 21 13:23:43.493100 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:23:43.493106    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f004a996-efff-468e-8501-e464d98ee25d-home\") pod \"f004a996-efff-468e-8501-e464d98ee25d\" (UID: \"f004a996-efff-468e-8501-e464d98ee25d\") "
Jul 21 13:23:43.493361 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:23:43.493148    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h5flq\" (UniqueName: \"kubernetes.io/projected/f004a996-efff-468e-8501-e464d98ee25d-kube-api-access-h5flq\") pod \"f004a996-efff-468e-8501-e464d98ee25d\" (UID: \"f004a996-efff-468e-8501-e464d98ee25d\") "
Jul 21 13:23:43.493361 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:23:43.493184    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f004a996-efff-468e-8501-e464d98ee25d-model-cache\") pod \"f004a996-efff-468e-8501-e464d98ee25d\" (UID: \"f004a996-efff-468e-8501-e464d98ee25d\") "
Jul 21 13:23:43.493361 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:23:43.493312    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f004a996-efff-468e-8501-e464d98ee25d-tls-certs\") pod \"f004a996-efff-468e-8501-e464d98ee25d\" (UID: \"f004a996-efff-468e-8501-e464d98ee25d\") "
Jul 21 13:23:43.493524 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:23:43.493364    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f004a996-efff-468e-8501-e464d98ee25d-tmp-dir\") pod \"f004a996-efff-468e-8501-e464d98ee25d\" (UID: \"f004a996-efff-468e-8501-e464d98ee25d\") "
Jul 21 13:23:43.493524 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:23:43.493393    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f004a996-efff-468e-8501-e464d98ee25d-home" (OuterVolumeSpecName: "home") pod "f004a996-efff-468e-8501-e464d98ee25d" (UID: "f004a996-efff-468e-8501-e464d98ee25d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 13:23:43.493524 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:23:43.493507    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f004a996-efff-468e-8501-e464d98ee25d-model-cache" (OuterVolumeSpecName: "model-cache") pod "f004a996-efff-468e-8501-e464d98ee25d" (UID: "f004a996-efff-468e-8501-e464d98ee25d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 13:23:43.493730 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:23:43.493691    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f004a996-efff-468e-8501-e464d98ee25d-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "f004a996-efff-468e-8501-e464d98ee25d" (UID: "f004a996-efff-468e-8501-e464d98ee25d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 13:23:43.494068 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:23:43.493756    2556 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f004a996-efff-468e-8501-e464d98ee25d-home\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Jul 21 13:23:43.494068 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:23:43.493776    2556 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f004a996-efff-468e-8501-e464d98ee25d-model-cache\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Jul 21 13:23:43.495579 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:23:43.495549    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f004a996-efff-468e-8501-e464d98ee25d-kube-api-access-h5flq" (OuterVolumeSpecName: "kube-api-access-h5flq") pod "f004a996-efff-468e-8501-e464d98ee25d" (UID: "f004a996-efff-468e-8501-e464d98ee25d"). InnerVolumeSpecName "kube-api-access-h5flq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 13:23:43.495686 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:23:43.495588    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f004a996-efff-468e-8501-e464d98ee25d-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f004a996-efff-468e-8501-e464d98ee25d" (UID: "f004a996-efff-468e-8501-e464d98ee25d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 13:23:43.495686 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:23:43.495610    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f004a996-efff-468e-8501-e464d98ee25d-dshm" (OuterVolumeSpecName: "dshm") pod "f004a996-efff-468e-8501-e464d98ee25d" (UID: "f004a996-efff-468e-8501-e464d98ee25d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 13:23:43.594978 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:23:43.594941    2556 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f004a996-efff-468e-8501-e464d98ee25d-tmp-dir\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Jul 21 13:23:43.594978 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:23:43.594975    2556 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f004a996-efff-468e-8501-e464d98ee25d-dshm\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Jul 21 13:23:43.595203 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:23:43.594990    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h5flq\" (UniqueName: \"kubernetes.io/projected/f004a996-efff-468e-8501-e464d98ee25d-kube-api-access-h5flq\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Jul 21 13:23:43.595203 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:23:43.595005    2556 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f004a996-efff-468e-8501-e464d98ee25d-tls-certs\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Jul 21 13:23:44.057313 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:23:44.057278    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-b3a013/sub-test-model-b3a013-kserve-6b9bd9fbc5-wp8p9"]
Jul 21 13:23:44.057755 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:23:44.057743    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f004a996-efff-468e-8501-e464d98ee25d" containerName="main"
Jul 21 13:23:44.057810 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:23:44.057757    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="f004a996-efff-468e-8501-e464d98ee25d" containerName="main"
Jul 21 13:23:44.057857 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:23:44.057847    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="f004a996-efff-468e-8501-e464d98ee25d" containerName="main"
Jul 21 13:23:44.061103 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:23:44.061086    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-b3a013/sub-test-model-b3a013-kserve-6b9bd9fbc5-wp8p9"
Jul 21 13:23:44.063264 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:23:44.063244    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-b3a013\"/\"sub-test-model-b3a013-kserve-self-signed-certs\""
Jul 21 13:23:44.070453 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:23:44.070430    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-b3a013/sub-test-model-b3a013-kserve-6b9bd9fbc5-wp8p9"]
Jul 21 13:23:44.099844 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:23:44.099816    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e0b69550-3a1d-4c5e-8438-e9df2e2fa7a3-model-cache\") pod \"sub-test-model-b3a013-kserve-6b9bd9fbc5-wp8p9\" (UID: \"e0b69550-3a1d-4c5e-8438-e9df2e2fa7a3\") " pod="ai-tenant-e2e-shared-a-b3a013/sub-test-model-b3a013-kserve-6b9bd9fbc5-wp8p9"
Jul 21 13:23:44.099973 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:23:44.099858    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e0b69550-3a1d-4c5e-8438-e9df2e2fa7a3-home\") pod \"sub-test-model-b3a013-kserve-6b9bd9fbc5-wp8p9\" (UID: \"e0b69550-3a1d-4c5e-8438-e9df2e2fa7a3\") " pod="ai-tenant-e2e-shared-a-b3a013/sub-test-model-b3a013-kserve-6b9bd9fbc5-wp8p9"
Jul 21 13:23:44.099973 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:23:44.099893    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e0b69550-3a1d-4c5e-8438-e9df2e2fa7a3-dshm\") pod \"sub-test-model-b3a013-kserve-6b9bd9fbc5-wp8p9\" (UID: \"e0b69550-3a1d-4c5e-8438-e9df2e2fa7a3\") " pod="ai-tenant-e2e-shared-a-b3a013/sub-test-model-b3a013-kserve-6b9bd9fbc5-wp8p9"
Jul 21 13:23:44.099973 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:23:44.099964    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e0b69550-3a1d-4c5e-8438-e9df2e2fa7a3-tls-certs\") pod \"sub-test-model-b3a013-kserve-6b9bd9fbc5-wp8p9\" (UID: \"e0b69550-3a1d-4c5e-8438-e9df2e2fa7a3\") " pod="ai-tenant-e2e-shared-a-b3a013/sub-test-model-b3a013-kserve-6b9bd9fbc5-wp8p9"
Jul 21 13:23:44.100165 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:23:44.100078    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e0b69550-3a1d-4c5e-8438-e9df2e2fa7a3-tmp-dir\") pod \"sub-test-model-b3a013-kserve-6b9bd9fbc5-wp8p9\" (UID: \"e0b69550-3a1d-4c5e-8438-e9df2e2fa7a3\") " pod="ai-tenant-e2e-shared-a-b3a013/sub-test-model-b3a013-kserve-6b9bd9fbc5-wp8p9"
Jul 21 13:23:44.100165 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:23:44.100130    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xlp56\" (UniqueName: \"kubernetes.io/projected/e0b69550-3a1d-4c5e-8438-e9df2e2fa7a3-kube-api-access-xlp56\") pod \"sub-test-model-b3a013-kserve-6b9bd9fbc5-wp8p9\" (UID: \"e0b69550-3a1d-4c5e-8438-e9df2e2fa7a3\") " pod="ai-tenant-e2e-shared-a-b3a013/sub-test-model-b3a013-kserve-6b9bd9fbc5-wp8p9"
Jul 21 13:23:44.147410 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:23:44.147379    2556 generic.go:356] "Generic (PLEG): container finished" podID="f004a996-efff-468e-8501-e464d98ee25d" containerID="d2f8dc0789f96f553f0f57a3c9924aaed8d9058dd74087ee64ac6854a0ea95f2" exitCode=0
Jul 21 13:23:44.147563 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:23:44.147448    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-b3a013/auth-test-model-b3a013-kserve-79d7f98b99-qfrjp" event={"ID":"f004a996-efff-468e-8501-e464d98ee25d","Type":"ContainerDied","Data":"d2f8dc0789f96f553f0f57a3c9924aaed8d9058dd74087ee64ac6854a0ea95f2"}
Jul 21 13:23:44.147563 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:23:44.147484    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-b3a013/auth-test-model-b3a013-kserve-79d7f98b99-qfrjp"
Jul 21 13:23:44.147563 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:23:44.147501    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-b3a013/auth-test-model-b3a013-kserve-79d7f98b99-qfrjp" event={"ID":"f004a996-efff-468e-8501-e464d98ee25d","Type":"ContainerDied","Data":"36bd7913823494c156aa7c6367f9a5b813460ee6f86e68bed44d368dd9e4dedc"}
Jul 21 13:23:44.147563 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:23:44.147526    2556 scope.go:117] "RemoveContainer" containerID="d2f8dc0789f96f553f0f57a3c9924aaed8d9058dd74087ee64ac6854a0ea95f2"
Jul 21 13:23:44.157646 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:23:44.157452    2556 scope.go:117] "RemoveContainer" containerID="d2f8dc0789f96f553f0f57a3c9924aaed8d9058dd74087ee64ac6854a0ea95f2"
Jul 21 13:23:44.157870 ip-10-0-130-2 kubenswrapper[2556]: E0721 13:23:44.157707    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d2f8dc0789f96f553f0f57a3c9924aaed8d9058dd74087ee64ac6854a0ea95f2\": container with ID starting with d2f8dc0789f96f553f0f57a3c9924aaed8d9058dd74087ee64ac6854a0ea95f2 not found: ID does not exist" containerID="d2f8dc0789f96f553f0f57a3c9924aaed8d9058dd74087ee64ac6854a0ea95f2"
Jul 21 13:23:44.157870 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:23:44.157729    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d2f8dc0789f96f553f0f57a3c9924aaed8d9058dd74087ee64ac6854a0ea95f2"} err="failed to get container status \"d2f8dc0789f96f553f0f57a3c9924aaed8d9058dd74087ee64ac6854a0ea95f2\": rpc error: code = NotFound desc = could not find container \"d2f8dc0789f96f553f0f57a3c9924aaed8d9058dd74087ee64ac6854a0ea95f2\": container with ID starting with d2f8dc0789f96f553f0f57a3c9924aaed8d9058dd74087ee64ac6854a0ea95f2 not found: ID does not exist"
Jul 21 13:23:44.170332 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:23:44.170309    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-b3a013/auth-test-model-b3a013-kserve-79d7f98b99-qfrjp"]
Jul 21 13:23:44.172485 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:23:44.172468    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-b3a013/auth-test-model-b3a013-kserve-79d7f98b99-qfrjp"]
Jul 21 13:23:44.201053 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:23:44.201004    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e0b69550-3a1d-4c5e-8438-e9df2e2fa7a3-home\") pod \"sub-test-model-b3a013-kserve-6b9bd9fbc5-wp8p9\" (UID: \"e0b69550-3a1d-4c5e-8438-e9df2e2fa7a3\") " pod="ai-tenant-e2e-shared-a-b3a013/sub-test-model-b3a013-kserve-6b9bd9fbc5-wp8p9"
Jul 21 13:23:44.201164 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:23:44.201070    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e0b69550-3a1d-4c5e-8438-e9df2e2fa7a3-dshm\") pod \"sub-test-model-b3a013-kserve-6b9bd9fbc5-wp8p9\" (UID: \"e0b69550-3a1d-4c5e-8438-e9df2e2fa7a3\") " pod="ai-tenant-e2e-shared-a-b3a013/sub-test-model-b3a013-kserve-6b9bd9fbc5-wp8p9"
Jul 21 13:23:44.201164 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:23:44.201105    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e0b69550-3a1d-4c5e-8438-e9df2e2fa7a3-tls-certs\") pod \"sub-test-model-b3a013-kserve-6b9bd9fbc5-wp8p9\" (UID: \"e0b69550-3a1d-4c5e-8438-e9df2e2fa7a3\") " pod="ai-tenant-e2e-shared-a-b3a013/sub-test-model-b3a013-kserve-6b9bd9fbc5-wp8p9"
Jul 21 13:23:44.201164 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:23:44.201151    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e0b69550-3a1d-4c5e-8438-e9df2e2fa7a3-tmp-dir\") pod \"sub-test-model-b3a013-kserve-6b9bd9fbc5-wp8p9\" (UID: \"e0b69550-3a1d-4c5e-8438-e9df2e2fa7a3\") " pod="ai-tenant-e2e-shared-a-b3a013/sub-test-model-b3a013-kserve-6b9bd9fbc5-wp8p9"
Jul 21 13:23:44.201335 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:23:44.201190    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xlp56\" (UniqueName: \"kubernetes.io/projected/e0b69550-3a1d-4c5e-8438-e9df2e2fa7a3-kube-api-access-xlp56\") pod \"sub-test-model-b3a013-kserve-6b9bd9fbc5-wp8p9\" (UID: \"e0b69550-3a1d-4c5e-8438-e9df2e2fa7a3\") " pod="ai-tenant-e2e-shared-a-b3a013/sub-test-model-b3a013-kserve-6b9bd9fbc5-wp8p9"
Jul 21 13:23:44.201335 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:23:44.201235    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e0b69550-3a1d-4c5e-8438-e9df2e2fa7a3-model-cache\") pod \"sub-test-model-b3a013-kserve-6b9bd9fbc5-wp8p9\" (UID: \"e0b69550-3a1d-4c5e-8438-e9df2e2fa7a3\") " pod="ai-tenant-e2e-shared-a-b3a013/sub-test-model-b3a013-kserve-6b9bd9fbc5-wp8p9"
Jul 21 13:23:44.201441 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:23:44.201371    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e0b69550-3a1d-4c5e-8438-e9df2e2fa7a3-home\") pod \"sub-test-model-b3a013-kserve-6b9bd9fbc5-wp8p9\" (UID: \"e0b69550-3a1d-4c5e-8438-e9df2e2fa7a3\") " pod="ai-tenant-e2e-shared-a-b3a013/sub-test-model-b3a013-kserve-6b9bd9fbc5-wp8p9"
Jul 21 13:23:44.201498 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:23:44.201469    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e0b69550-3a1d-4c5e-8438-e9df2e2fa7a3-tmp-dir\") pod \"sub-test-model-b3a013-kserve-6b9bd9fbc5-wp8p9\" (UID: \"e0b69550-3a1d-4c5e-8438-e9df2e2fa7a3\") " pod="ai-tenant-e2e-shared-a-b3a013/sub-test-model-b3a013-kserve-6b9bd9fbc5-wp8p9"
Jul 21 13:23:44.201561 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:23:44.201544    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e0b69550-3a1d-4c5e-8438-e9df2e2fa7a3-model-cache\") pod \"sub-test-model-b3a013-kserve-6b9bd9fbc5-wp8p9\" (UID: \"e0b69550-3a1d-4c5e-8438-e9df2e2fa7a3\") " pod="ai-tenant-e2e-shared-a-b3a013/sub-test-model-b3a013-kserve-6b9bd9fbc5-wp8p9"
Jul 21 13:23:44.203390 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:23:44.203364    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e0b69550-3a1d-4c5e-8438-e9df2e2fa7a3-dshm\") pod \"sub-test-model-b3a013-kserve-6b9bd9fbc5-wp8p9\" (UID: \"e0b69550-3a1d-4c5e-8438-e9df2e2fa7a3\") " pod="ai-tenant-e2e-shared-a-b3a013/sub-test-model-b3a013-kserve-6b9bd9fbc5-wp8p9"
Jul 21 13:23:44.203845 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:23:44.203822    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e0b69550-3a1d-4c5e-8438-e9df2e2fa7a3-tls-certs\") pod \"sub-test-model-b3a013-kserve-6b9bd9fbc5-wp8p9\" (UID: \"e0b69550-3a1d-4c5e-8438-e9df2e2fa7a3\") " pod="ai-tenant-e2e-shared-a-b3a013/sub-test-model-b3a013-kserve-6b9bd9fbc5-wp8p9"
Jul 21 13:23:44.210442 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:23:44.210417    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xlp56\" (UniqueName: \"kubernetes.io/projected/e0b69550-3a1d-4c5e-8438-e9df2e2fa7a3-kube-api-access-xlp56\") pod \"sub-test-model-b3a013-kserve-6b9bd9fbc5-wp8p9\" (UID: \"e0b69550-3a1d-4c5e-8438-e9df2e2fa7a3\") " pod="ai-tenant-e2e-shared-a-b3a013/sub-test-model-b3a013-kserve-6b9bd9fbc5-wp8p9"
Jul 21 13:23:44.371132 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:23:44.371048    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-b3a013/sub-test-model-b3a013-kserve-6b9bd9fbc5-wp8p9"
Jul 21 13:23:44.522553 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:23:44.522523    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-b3a013/sub-test-model-b3a013-kserve-6b9bd9fbc5-wp8p9"]
Jul 21 13:23:44.524228 ip-10-0-130-2 kubenswrapper[2556]: W0721 13:23:44.524199    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode0b69550_3a1d_4c5e_8438_e9df2e2fa7a3.slice/crio-459605ac3dabbeadd456be69e54730586d0b284fbbe75a01be6ffe6ae92539e5 WatchSource:0}: Error finding container 459605ac3dabbeadd456be69e54730586d0b284fbbe75a01be6ffe6ae92539e5: Status 404 returned error can't find the container with id 459605ac3dabbeadd456be69e54730586d0b284fbbe75a01be6ffe6ae92539e5
Jul 21 13:23:45.155252 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:23:45.155217    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-b3a013/sub-test-model-b3a013-kserve-6b9bd9fbc5-wp8p9" event={"ID":"e0b69550-3a1d-4c5e-8438-e9df2e2fa7a3","Type":"ContainerStarted","Data":"892efb2aa9d277b95ff4cc0a3d1edd5b030b90a9035efe3ebeafb2e7649ed9e7"}
Jul 21 13:23:45.155252 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:23:45.155260    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-b3a013/sub-test-model-b3a013-kserve-6b9bd9fbc5-wp8p9" event={"ID":"e0b69550-3a1d-4c5e-8438-e9df2e2fa7a3","Type":"ContainerStarted","Data":"459605ac3dabbeadd456be69e54730586d0b284fbbe75a01be6ffe6ae92539e5"}
Jul 21 13:23:45.173136 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:23:45.173089    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-b3a013/sub-test-model-b3a013-kserve-6b9bd9fbc5-wp8p9" podStartSLOduration=1.173074423 podStartE2EDuration="1.173074423s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 13:23:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:23:45.172235593 +0000 UTC m=+2638.476240694" watchObservedRunningTime="2026-07-21 13:23:45.173074423 +0000 UTC m=+2638.477079504"
Jul 21 13:23:45.204752 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:23:45.204716    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f004a996-efff-468e-8501-e464d98ee25d" path="/var/lib/kubelet/pods/f004a996-efff-468e-8501-e464d98ee25d/volumes"
Jul 21 13:23:54.371497 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:23:54.371463    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-b3a013/sub-test-model-b3a013-kserve-6b9bd9fbc5-wp8p9"
Jul 21 13:23:54.371865 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:23:54.371516    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-b3a013/sub-test-model-b3a013-kserve-6b9bd9fbc5-wp8p9"
Jul 21 13:23:54.384125 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:23:54.384103    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-b3a013/sub-test-model-b3a013-kserve-6b9bd9fbc5-wp8p9"
Jul 21 13:23:55.220807 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:23:55.220764    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-b3a013/sub-test-model-b3a013-kserve-6b9bd9fbc5-wp8p9"
Jul 21 13:24:51.586671 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:24:51.586639    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-b3a013/sub-test-model-b3a013-kserve-6b9bd9fbc5-wp8p9"]
Jul 21 13:25:06.607514 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:25:06.607440    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-b3a013/sub-test-model-b3a013-kserve-6b9bd9fbc5-wp8p9" podUID="e0b69550-3a1d-4c5e-8438-e9df2e2fa7a3" containerName="main" containerID="cri-o://892efb2aa9d277b95ff4cc0a3d1edd5b030b90a9035efe3ebeafb2e7649ed9e7" gracePeriod=45
Jul 21 13:25:06.868518 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:25:06.868467    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-b3a013/sub-test-model-b3a013-kserve-6b9bd9fbc5-wp8p9"
Jul 21 13:25:06.990915 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:25:06.990889    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e0b69550-3a1d-4c5e-8438-e9df2e2fa7a3-tls-certs\") pod \"e0b69550-3a1d-4c5e-8438-e9df2e2fa7a3\" (UID: \"e0b69550-3a1d-4c5e-8438-e9df2e2fa7a3\") "
Jul 21 13:25:06.991091 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:25:06.990921    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e0b69550-3a1d-4c5e-8438-e9df2e2fa7a3-home\") pod \"e0b69550-3a1d-4c5e-8438-e9df2e2fa7a3\" (UID: \"e0b69550-3a1d-4c5e-8438-e9df2e2fa7a3\") "
Jul 21 13:25:06.991091 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:25:06.990950    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e0b69550-3a1d-4c5e-8438-e9df2e2fa7a3-model-cache\") pod \"e0b69550-3a1d-4c5e-8438-e9df2e2fa7a3\" (UID: \"e0b69550-3a1d-4c5e-8438-e9df2e2fa7a3\") "
Jul 21 13:25:06.991091 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:25:06.990965    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e0b69550-3a1d-4c5e-8438-e9df2e2fa7a3-dshm\") pod \"e0b69550-3a1d-4c5e-8438-e9df2e2fa7a3\" (UID: \"e0b69550-3a1d-4c5e-8438-e9df2e2fa7a3\") "
Jul 21 13:25:06.991091 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:25:06.991009    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xlp56\" (UniqueName: \"kubernetes.io/projected/e0b69550-3a1d-4c5e-8438-e9df2e2fa7a3-kube-api-access-xlp56\") pod \"e0b69550-3a1d-4c5e-8438-e9df2e2fa7a3\" (UID: \"e0b69550-3a1d-4c5e-8438-e9df2e2fa7a3\") "
Jul 21 13:25:06.991313 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:25:06.991103    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e0b69550-3a1d-4c5e-8438-e9df2e2fa7a3-tmp-dir\") pod \"e0b69550-3a1d-4c5e-8438-e9df2e2fa7a3\" (UID: \"e0b69550-3a1d-4c5e-8438-e9df2e2fa7a3\") "
Jul 21 13:25:06.991313 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:25:06.991233    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e0b69550-3a1d-4c5e-8438-e9df2e2fa7a3-model-cache" (OuterVolumeSpecName: "model-cache") pod "e0b69550-3a1d-4c5e-8438-e9df2e2fa7a3" (UID: "e0b69550-3a1d-4c5e-8438-e9df2e2fa7a3"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 13:25:06.991313 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:25:06.991245    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e0b69550-3a1d-4c5e-8438-e9df2e2fa7a3-home" (OuterVolumeSpecName: "home") pod "e0b69550-3a1d-4c5e-8438-e9df2e2fa7a3" (UID: "e0b69550-3a1d-4c5e-8438-e9df2e2fa7a3"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 13:25:06.991526 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:25:06.991505    2556 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e0b69550-3a1d-4c5e-8438-e9df2e2fa7a3-home\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Jul 21 13:25:06.991628 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:25:06.991531    2556 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e0b69550-3a1d-4c5e-8438-e9df2e2fa7a3-model-cache\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Jul 21 13:25:06.991628 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:25:06.991505    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e0b69550-3a1d-4c5e-8438-e9df2e2fa7a3-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "e0b69550-3a1d-4c5e-8438-e9df2e2fa7a3" (UID: "e0b69550-3a1d-4c5e-8438-e9df2e2fa7a3"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 13:25:06.993422 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:25:06.993396    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e0b69550-3a1d-4c5e-8438-e9df2e2fa7a3-kube-api-access-xlp56" (OuterVolumeSpecName: "kube-api-access-xlp56") pod "e0b69550-3a1d-4c5e-8438-e9df2e2fa7a3" (UID: "e0b69550-3a1d-4c5e-8438-e9df2e2fa7a3"). InnerVolumeSpecName "kube-api-access-xlp56". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 13:25:06.993663 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:25:06.993636    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e0b69550-3a1d-4c5e-8438-e9df2e2fa7a3-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e0b69550-3a1d-4c5e-8438-e9df2e2fa7a3" (UID: "e0b69550-3a1d-4c5e-8438-e9df2e2fa7a3"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 13:25:06.993663 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:25:06.993638    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e0b69550-3a1d-4c5e-8438-e9df2e2fa7a3-dshm" (OuterVolumeSpecName: "dshm") pod "e0b69550-3a1d-4c5e-8438-e9df2e2fa7a3" (UID: "e0b69550-3a1d-4c5e-8438-e9df2e2fa7a3"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 13:25:07.092808 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:25:07.092782    2556 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e0b69550-3a1d-4c5e-8438-e9df2e2fa7a3-dshm\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Jul 21 13:25:07.092808 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:25:07.092806    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xlp56\" (UniqueName: \"kubernetes.io/projected/e0b69550-3a1d-4c5e-8438-e9df2e2fa7a3-kube-api-access-xlp56\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Jul 21 13:25:07.092933 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:25:07.092835    2556 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e0b69550-3a1d-4c5e-8438-e9df2e2fa7a3-tmp-dir\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Jul 21 13:25:07.092933 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:25:07.092845    2556 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e0b69550-3a1d-4c5e-8438-e9df2e2fa7a3-tls-certs\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Jul 21 13:25:07.619748 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:25:07.619715    2556 generic.go:356] "Generic (PLEG): container finished" podID="e0b69550-3a1d-4c5e-8438-e9df2e2fa7a3" containerID="892efb2aa9d277b95ff4cc0a3d1edd5b030b90a9035efe3ebeafb2e7649ed9e7" exitCode=0
Jul 21 13:25:07.620132 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:25:07.619757    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-b3a013/sub-test-model-b3a013-kserve-6b9bd9fbc5-wp8p9"
Jul 21 13:25:07.620132 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:25:07.619790    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-b3a013/sub-test-model-b3a013-kserve-6b9bd9fbc5-wp8p9" event={"ID":"e0b69550-3a1d-4c5e-8438-e9df2e2fa7a3","Type":"ContainerDied","Data":"892efb2aa9d277b95ff4cc0a3d1edd5b030b90a9035efe3ebeafb2e7649ed9e7"}
Jul 21 13:25:07.620132 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:25:07.619827    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-b3a013/sub-test-model-b3a013-kserve-6b9bd9fbc5-wp8p9" event={"ID":"e0b69550-3a1d-4c5e-8438-e9df2e2fa7a3","Type":"ContainerDied","Data":"459605ac3dabbeadd456be69e54730586d0b284fbbe75a01be6ffe6ae92539e5"}
Jul 21 13:25:07.620132 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:25:07.619842    2556 scope.go:117] "RemoveContainer" containerID="892efb2aa9d277b95ff4cc0a3d1edd5b030b90a9035efe3ebeafb2e7649ed9e7"
Jul 21 13:25:07.628993 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:25:07.628975    2556 scope.go:117] "RemoveContainer" containerID="892efb2aa9d277b95ff4cc0a3d1edd5b030b90a9035efe3ebeafb2e7649ed9e7"
Jul 21 13:25:07.629278 ip-10-0-130-2 kubenswrapper[2556]: E0721 13:25:07.629260    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"892efb2aa9d277b95ff4cc0a3d1edd5b030b90a9035efe3ebeafb2e7649ed9e7\": container with ID starting with 892efb2aa9d277b95ff4cc0a3d1edd5b030b90a9035efe3ebeafb2e7649ed9e7 not found: ID does not exist" containerID="892efb2aa9d277b95ff4cc0a3d1edd5b030b90a9035efe3ebeafb2e7649ed9e7"
Jul 21 13:25:07.629341 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:25:07.629287    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"892efb2aa9d277b95ff4cc0a3d1edd5b030b90a9035efe3ebeafb2e7649ed9e7"} err="failed to get container status \"892efb2aa9d277b95ff4cc0a3d1edd5b030b90a9035efe3ebeafb2e7649ed9e7\": rpc error: code = NotFound desc = could not find container \"892efb2aa9d277b95ff4cc0a3d1edd5b030b90a9035efe3ebeafb2e7649ed9e7\": container with ID starting with 892efb2aa9d277b95ff4cc0a3d1edd5b030b90a9035efe3ebeafb2e7649ed9e7 not found: ID does not exist"
Jul 21 13:25:07.638379 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:25:07.638354    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-b3a013/sub-test-model-b3a013-kserve-6b9bd9fbc5-wp8p9"]
Jul 21 13:25:07.644833 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:25:07.644814    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-b3a013/sub-test-model-b3a013-kserve-6b9bd9fbc5-wp8p9"]
Jul 21 13:25:09.194617 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:25:09.194585    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e0b69550-3a1d-4c5e-8438-e9df2e2fa7a3" path="/var/lib/kubelet/pods/e0b69550-3a1d-4c5e-8438-e9df2e2fa7a3/volumes"
Jul 21 13:25:36.668928 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:25:36.668886    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-cfed01/rate-test-model-cfed01-kserve-567c966cf5-df9ch"]
Jul 21 13:25:36.669657 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:25:36.669632    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e0b69550-3a1d-4c5e-8438-e9df2e2fa7a3" containerName="main"
Jul 21 13:25:36.669657 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:25:36.669659    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="e0b69550-3a1d-4c5e-8438-e9df2e2fa7a3" containerName="main"
Jul 21 13:25:36.669816 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:25:36.669779    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="e0b69550-3a1d-4c5e-8438-e9df2e2fa7a3" containerName="main"
Jul 21 13:25:36.673652 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:25:36.673631    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-cfed01/rate-test-model-cfed01-kserve-567c966cf5-df9ch"
Jul 21 13:25:36.676252 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:25:36.676224    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-cfed01\"/\"openshift-service-ca.crt\""
Jul 21 13:25:36.676693 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:25:36.676673    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-cfed01\"/\"rate-test-model-cfed01-kserve-self-signed-certs\""
Jul 21 13:25:36.676800 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:25:36.676680    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-cfed01\"/\"kube-root-ca.crt\""
Jul 21 13:25:36.676865 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:25:36.676814    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-cfed01\"/\"default-dockercfg-4qzr4\""
Jul 21 13:25:36.683905 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:25:36.683882    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-cfed01/rate-test-model-cfed01-kserve-567c966cf5-df9ch"]
Jul 21 13:25:36.747967 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:25:36.747925    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/60c99958-b23f-4814-8db7-04c29b8e8d03-tmp-dir\") pod \"rate-test-model-cfed01-kserve-567c966cf5-df9ch\" (UID: \"60c99958-b23f-4814-8db7-04c29b8e8d03\") " pod="ai-tenant-e2e-shared-b-cfed01/rate-test-model-cfed01-kserve-567c966cf5-df9ch"
Jul 21 13:25:36.748161 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:25:36.747987    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/60c99958-b23f-4814-8db7-04c29b8e8d03-model-cache\") pod \"rate-test-model-cfed01-kserve-567c966cf5-df9ch\" (UID: \"60c99958-b23f-4814-8db7-04c29b8e8d03\") " pod="ai-tenant-e2e-shared-b-cfed01/rate-test-model-cfed01-kserve-567c966cf5-df9ch"
Jul 21 13:25:36.748161 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:25:36.748032    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/60c99958-b23f-4814-8db7-04c29b8e8d03-home\") pod \"rate-test-model-cfed01-kserve-567c966cf5-df9ch\" (UID: \"60c99958-b23f-4814-8db7-04c29b8e8d03\") " pod="ai-tenant-e2e-shared-b-cfed01/rate-test-model-cfed01-kserve-567c966cf5-df9ch"
Jul 21 13:25:36.748161 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:25:36.748055    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/60c99958-b23f-4814-8db7-04c29b8e8d03-dshm\") pod \"rate-test-model-cfed01-kserve-567c966cf5-df9ch\" (UID: \"60c99958-b23f-4814-8db7-04c29b8e8d03\") " pod="ai-tenant-e2e-shared-b-cfed01/rate-test-model-cfed01-kserve-567c966cf5-df9ch"
Jul 21 13:25:36.748161 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:25:36.748140    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/60c99958-b23f-4814-8db7-04c29b8e8d03-tls-certs\") pod \"rate-test-model-cfed01-kserve-567c966cf5-df9ch\" (UID: \"60c99958-b23f-4814-8db7-04c29b8e8d03\") " pod="ai-tenant-e2e-shared-b-cfed01/rate-test-model-cfed01-kserve-567c966cf5-df9ch"
Jul 21 13:25:36.748346 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:25:36.748175    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rd7lx\" (UniqueName: \"kubernetes.io/projected/60c99958-b23f-4814-8db7-04c29b8e8d03-kube-api-access-rd7lx\") pod \"rate-test-model-cfed01-kserve-567c966cf5-df9ch\" (UID: \"60c99958-b23f-4814-8db7-04c29b8e8d03\") " pod="ai-tenant-e2e-shared-b-cfed01/rate-test-model-cfed01-kserve-567c966cf5-df9ch"
Jul 21 13:25:36.849329 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:25:36.849278    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/60c99958-b23f-4814-8db7-04c29b8e8d03-tmp-dir\") pod \"rate-test-model-cfed01-kserve-567c966cf5-df9ch\" (UID: \"60c99958-b23f-4814-8db7-04c29b8e8d03\") " pod="ai-tenant-e2e-shared-b-cfed01/rate-test-model-cfed01-kserve-567c966cf5-df9ch"
Jul 21 13:25:36.849527 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:25:36.849357    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/60c99958-b23f-4814-8db7-04c29b8e8d03-model-cache\") pod \"rate-test-model-cfed01-kserve-567c966cf5-df9ch\" (UID: \"60c99958-b23f-4814-8db7-04c29b8e8d03\") " pod="ai-tenant-e2e-shared-b-cfed01/rate-test-model-cfed01-kserve-567c966cf5-df9ch"
Jul 21 13:25:36.849527 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:25:36.849380    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/60c99958-b23f-4814-8db7-04c29b8e8d03-home\") pod \"rate-test-model-cfed01-kserve-567c966cf5-df9ch\" (UID: \"60c99958-b23f-4814-8db7-04c29b8e8d03\") " pod="ai-tenant-e2e-shared-b-cfed01/rate-test-model-cfed01-kserve-567c966cf5-df9ch"
Jul 21 13:25:36.849527 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:25:36.849395    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/60c99958-b23f-4814-8db7-04c29b8e8d03-dshm\") pod \"rate-test-model-cfed01-kserve-567c966cf5-df9ch\" (UID: \"60c99958-b23f-4814-8db7-04c29b8e8d03\") " pod="ai-tenant-e2e-shared-b-cfed01/rate-test-model-cfed01-kserve-567c966cf5-df9ch"
Jul 21 13:25:36.849527 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:25:36.849439    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/60c99958-b23f-4814-8db7-04c29b8e8d03-tls-certs\") pod \"rate-test-model-cfed01-kserve-567c966cf5-df9ch\" (UID: \"60c99958-b23f-4814-8db7-04c29b8e8d03\") " pod="ai-tenant-e2e-shared-b-cfed01/rate-test-model-cfed01-kserve-567c966cf5-df9ch"
Jul 21 13:25:36.849527 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:25:36.849496    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rd7lx\" (UniqueName: \"kubernetes.io/projected/60c99958-b23f-4814-8db7-04c29b8e8d03-kube-api-access-rd7lx\") pod \"rate-test-model-cfed01-kserve-567c966cf5-df9ch\" (UID: \"60c99958-b23f-4814-8db7-04c29b8e8d03\") " pod="ai-tenant-e2e-shared-b-cfed01/rate-test-model-cfed01-kserve-567c966cf5-df9ch"
Jul 21 13:25:36.849776 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:25:36.849746    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/60c99958-b23f-4814-8db7-04c29b8e8d03-tmp-dir\") pod \"rate-test-model-cfed01-kserve-567c966cf5-df9ch\" (UID: \"60c99958-b23f-4814-8db7-04c29b8e8d03\") " pod="ai-tenant-e2e-shared-b-cfed01/rate-test-model-cfed01-kserve-567c966cf5-df9ch"
Jul 21 13:25:36.849856 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:25:36.849787    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/60c99958-b23f-4814-8db7-04c29b8e8d03-model-cache\") pod \"rate-test-model-cfed01-kserve-567c966cf5-df9ch\" (UID: \"60c99958-b23f-4814-8db7-04c29b8e8d03\") " pod="ai-tenant-e2e-shared-b-cfed01/rate-test-model-cfed01-kserve-567c966cf5-df9ch"
Jul 21 13:25:36.849856 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:25:36.849813    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/60c99958-b23f-4814-8db7-04c29b8e8d03-home\") pod \"rate-test-model-cfed01-kserve-567c966cf5-df9ch\" (UID: \"60c99958-b23f-4814-8db7-04c29b8e8d03\") " pod="ai-tenant-e2e-shared-b-cfed01/rate-test-model-cfed01-kserve-567c966cf5-df9ch"
Jul 21 13:25:36.851870 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:25:36.851845    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/60c99958-b23f-4814-8db7-04c29b8e8d03-dshm\") pod \"rate-test-model-cfed01-kserve-567c966cf5-df9ch\" (UID: \"60c99958-b23f-4814-8db7-04c29b8e8d03\") " pod="ai-tenant-e2e-shared-b-cfed01/rate-test-model-cfed01-kserve-567c966cf5-df9ch"
Jul 21 13:25:36.852029 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:25:36.851999    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/60c99958-b23f-4814-8db7-04c29b8e8d03-tls-certs\") pod \"rate-test-model-cfed01-kserve-567c966cf5-df9ch\" (UID: \"60c99958-b23f-4814-8db7-04c29b8e8d03\") " pod="ai-tenant-e2e-shared-b-cfed01/rate-test-model-cfed01-kserve-567c966cf5-df9ch"
Jul 21 13:25:36.858572 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:25:36.858548    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rd7lx\" (UniqueName: \"kubernetes.io/projected/60c99958-b23f-4814-8db7-04c29b8e8d03-kube-api-access-rd7lx\") pod \"rate-test-model-cfed01-kserve-567c966cf5-df9ch\" (UID: \"60c99958-b23f-4814-8db7-04c29b8e8d03\") " pod="ai-tenant-e2e-shared-b-cfed01/rate-test-model-cfed01-kserve-567c966cf5-df9ch"
Jul 21 13:25:36.985008 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:25:36.984976    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-cfed01/rate-test-model-cfed01-kserve-567c966cf5-df9ch"
Jul 21 13:25:37.134242 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:25:37.134217    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-cfed01/rate-test-model-cfed01-kserve-567c966cf5-df9ch"]
Jul 21 13:25:37.136519 ip-10-0-130-2 kubenswrapper[2556]: W0721 13:25:37.136495    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod60c99958_b23f_4814_8db7_04c29b8e8d03.slice/crio-6ef9ebbcb02398c92a5d088f072e3b434691fc6a50ba40b054747ad2d2c953f7 WatchSource:0}: Error finding container 6ef9ebbcb02398c92a5d088f072e3b434691fc6a50ba40b054747ad2d2c953f7: Status 404 returned error can't find the container with id 6ef9ebbcb02398c92a5d088f072e3b434691fc6a50ba40b054747ad2d2c953f7
Jul 21 13:25:37.794779 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:25:37.794743    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-cfed01/rate-test-model-cfed01-kserve-567c966cf5-df9ch" event={"ID":"60c99958-b23f-4814-8db7-04c29b8e8d03","Type":"ContainerStarted","Data":"b8d045b7dd28f4afafedc09a9a74d59b97e4a8b7daad4625596c78a81ebd1563"}
Jul 21 13:25:37.795198 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:25:37.794794    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-cfed01/rate-test-model-cfed01-kserve-567c966cf5-df9ch" event={"ID":"60c99958-b23f-4814-8db7-04c29b8e8d03","Type":"ContainerStarted","Data":"6ef9ebbcb02398c92a5d088f072e3b434691fc6a50ba40b054747ad2d2c953f7"}
Jul 21 13:25:37.819785 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:25:37.819735    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-cfed01/rate-test-model-cfed01-kserve-567c966cf5-df9ch" podStartSLOduration=1.819718883 podStartE2EDuration="1.819718883s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 13:25:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:25:37.817242317 +0000 UTC m=+2751.121247410" watchObservedRunningTime="2026-07-21 13:25:37.819718883 +0000 UTC m=+2751.123723972"
Jul 21 13:25:46.985411 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:25:46.985378    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-cfed01/rate-test-model-cfed01-kserve-567c966cf5-df9ch"
Jul 21 13:25:46.987860 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:25:46.985426    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-cfed01/rate-test-model-cfed01-kserve-567c966cf5-df9ch"
Jul 21 13:25:46.998031 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:25:46.997997    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-cfed01/rate-test-model-cfed01-kserve-567c966cf5-df9ch"
Jul 21 13:25:47.575990 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:25:47.575957    2556 scope.go:117] "RemoveContainer" containerID="9672be3e6498ab2e03c8b6681ccaf6c945d62129d17733778dfbdca7c5c55227"
Jul 21 13:25:47.864385 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:25:47.864305    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-cfed01/rate-test-model-cfed01-kserve-567c966cf5-df9ch"
Jul 21 13:26:17.750053 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:17.749954    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-cfed01/rate-test-model-cfed01-kserve-567c966cf5-df9ch"]
Jul 21 13:26:22.139415 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:22.139381    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7292-140084598090288"]
Jul 21 13:26:22.143185 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:22.143166    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7292-140084598090288"
Jul 21 13:26:22.145396 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:22.145372    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-bmbjm\""
Jul 21 13:26:22.150188 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:22.150152    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7292-140084598090288"]
Jul 21 13:26:22.288886 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:22.288854    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w4zck\" (UniqueName: \"kubernetes.io/projected/3f86c7f0-523e-4758-a349-6044f315c310-kube-api-access-w4zck\") pod \"test-curl-7292-140084598090288\" (UID: \"3f86c7f0-523e-4758-a349-6044f315c310\") " pod="opendatahub/test-curl-7292-140084598090288"
Jul 21 13:26:22.389818 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:22.389724    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w4zck\" (UniqueName: \"kubernetes.io/projected/3f86c7f0-523e-4758-a349-6044f315c310-kube-api-access-w4zck\") pod \"test-curl-7292-140084598090288\" (UID: \"3f86c7f0-523e-4758-a349-6044f315c310\") " pod="opendatahub/test-curl-7292-140084598090288"
Jul 21 13:26:22.399279 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:22.399249    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w4zck\" (UniqueName: \"kubernetes.io/projected/3f86c7f0-523e-4758-a349-6044f315c310-kube-api-access-w4zck\") pod \"test-curl-7292-140084598090288\" (UID: \"3f86c7f0-523e-4758-a349-6044f315c310\") " pod="opendatahub/test-curl-7292-140084598090288"
Jul 21 13:26:22.454555 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:22.454523    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7292-140084598090288"
Jul 21 13:26:22.595845 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:22.595818    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7292-140084598090288"]
Jul 21 13:26:22.598279 ip-10-0-130-2 kubenswrapper[2556]: W0721 13:26:22.598250    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3f86c7f0_523e_4758_a349_6044f315c310.slice/crio-a40ba3eb0734a78340a5230ccb6c834fa170a9c94708d67460145908d250e239 WatchSource:0}: Error finding container a40ba3eb0734a78340a5230ccb6c834fa170a9c94708d67460145908d250e239: Status 404 returned error can't find the container with id a40ba3eb0734a78340a5230ccb6c834fa170a9c94708d67460145908d250e239
Jul 21 13:26:22.599857 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:22.599829    2556 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 21 13:26:23.051659 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:23.051620    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7292-140084598090288" event={"ID":"3f86c7f0-523e-4758-a349-6044f315c310","Type":"ContainerStarted","Data":"a40ba3eb0734a78340a5230ccb6c834fa170a9c94708d67460145908d250e239"}
Jul 21 13:26:24.058592 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:24.058559    2556 generic.go:356] "Generic (PLEG): container finished" podID="3f86c7f0-523e-4758-a349-6044f315c310" containerID="959ff8ceb1a7c4b314f278ecbc8cf20b3917c92fedb04ae8b003534a26d57cf3" exitCode=0
Jul 21 13:26:24.058981 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:24.058646    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7292-140084598090288" event={"ID":"3f86c7f0-523e-4758-a349-6044f315c310","Type":"ContainerDied","Data":"959ff8ceb1a7c4b314f278ecbc8cf20b3917c92fedb04ae8b003534a26d57cf3"}
Jul 21 13:26:25.205907 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:25.205883    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7292-140084598090288"
Jul 21 13:26:25.254931 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:25.254900    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7292-140084598090288_3f86c7f0-523e-4758-a349-6044f315c310/test-curl-7292-140084598090288/0.log"
Jul 21 13:26:25.281239 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:25.281209    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7292-140084598090288"]
Jul 21 13:26:25.286053 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:25.286024    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7292-140084598090288"]
Jul 21 13:26:25.321284 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:25.321194    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w4zck\" (UniqueName: \"kubernetes.io/projected/3f86c7f0-523e-4758-a349-6044f315c310-kube-api-access-w4zck\") pod \"3f86c7f0-523e-4758-a349-6044f315c310\" (UID: \"3f86c7f0-523e-4758-a349-6044f315c310\") "
Jul 21 13:26:25.324313 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:25.324286    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3f86c7f0-523e-4758-a349-6044f315c310-kube-api-access-w4zck" (OuterVolumeSpecName: "kube-api-access-w4zck") pod "3f86c7f0-523e-4758-a349-6044f315c310" (UID: "3f86c7f0-523e-4758-a349-6044f315c310"). InnerVolumeSpecName "kube-api-access-w4zck". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 13:26:25.422496 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:25.422464    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w4zck\" (UniqueName: \"kubernetes.io/projected/3f86c7f0-523e-4758-a349-6044f315c310-kube-api-access-w4zck\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Jul 21 13:26:25.451826 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:25.451802    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7292-140084598023200"]
Jul 21 13:26:25.452267 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:25.452250    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3f86c7f0-523e-4758-a349-6044f315c310" containerName="test-curl-7292-140084598090288"
Jul 21 13:26:25.452316 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:25.452272    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="3f86c7f0-523e-4758-a349-6044f315c310" containerName="test-curl-7292-140084598090288"
Jul 21 13:26:25.452363 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:25.452353    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="3f86c7f0-523e-4758-a349-6044f315c310" containerName="test-curl-7292-140084598090288"
Jul 21 13:26:25.455438 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:25.455422    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7292-140084598023200"
Jul 21 13:26:25.463695 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:25.463672    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7292-140084598023200"]
Jul 21 13:26:25.623777 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:25.623670    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jqzmk\" (UniqueName: \"kubernetes.io/projected/7ad31adf-b6d9-4364-b7ea-a764f8274714-kube-api-access-jqzmk\") pod \"test-curl-7292-140084598023200\" (UID: \"7ad31adf-b6d9-4364-b7ea-a764f8274714\") " pod="opendatahub/test-curl-7292-140084598023200"
Jul 21 13:26:25.724793 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:25.724750    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jqzmk\" (UniqueName: \"kubernetes.io/projected/7ad31adf-b6d9-4364-b7ea-a764f8274714-kube-api-access-jqzmk\") pod \"test-curl-7292-140084598023200\" (UID: \"7ad31adf-b6d9-4364-b7ea-a764f8274714\") " pod="opendatahub/test-curl-7292-140084598023200"
Jul 21 13:26:25.736092 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:25.736067    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jqzmk\" (UniqueName: \"kubernetes.io/projected/7ad31adf-b6d9-4364-b7ea-a764f8274714-kube-api-access-jqzmk\") pod \"test-curl-7292-140084598023200\" (UID: \"7ad31adf-b6d9-4364-b7ea-a764f8274714\") " pod="opendatahub/test-curl-7292-140084598023200"
Jul 21 13:26:25.765141 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:25.765118    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7292-140084598023200"
Jul 21 13:26:25.904547 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:25.904525    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7292-140084598023200"]
Jul 21 13:26:25.906898 ip-10-0-130-2 kubenswrapper[2556]: W0721 13:26:25.906869    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7ad31adf_b6d9_4364_b7ea_a764f8274714.slice/crio-06058269e16f31ed6211e41b60b7d234296c2c4b14df8473b2f55d08a1d4c9b5 WatchSource:0}: Error finding container 06058269e16f31ed6211e41b60b7d234296c2c4b14df8473b2f55d08a1d4c9b5: Status 404 returned error can't find the container with id 06058269e16f31ed6211e41b60b7d234296c2c4b14df8473b2f55d08a1d4c9b5
Jul 21 13:26:26.070562 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:26.070532    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7292-140084598090288"
Jul 21 13:26:26.070720 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:26.070566    2556 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a40ba3eb0734a78340a5230ccb6c834fa170a9c94708d67460145908d250e239"
Jul 21 13:26:26.071787 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:26.071760    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7292-140084598023200" event={"ID":"7ad31adf-b6d9-4364-b7ea-a764f8274714","Type":"ContainerStarted","Data":"06058269e16f31ed6211e41b60b7d234296c2c4b14df8473b2f55d08a1d4c9b5"}
Jul 21 13:26:26.081338 ip-10-0-130-2 kubenswrapper[2556]: E0721 13:26:26.081315    2556 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7292-140084598090288\" is forbidden: User \"system:node:ip-10-0-130-2.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-130-2.ec2.internal' and this object" podUID="3f86c7f0-523e-4758-a349-6044f315c310" pod="opendatahub/test-curl-7292-140084598090288"
Jul 21 13:26:27.079136 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:27.079100    2556 generic.go:356] "Generic (PLEG): container finished" podID="7ad31adf-b6d9-4364-b7ea-a764f8274714" containerID="d4b58782057a94740c653ad27b6f84578efd92becabee7b2795f6dcffa479e6a" exitCode=0
Jul 21 13:26:27.079527 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:27.079193    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7292-140084598023200" event={"ID":"7ad31adf-b6d9-4364-b7ea-a764f8274714","Type":"ContainerDied","Data":"d4b58782057a94740c653ad27b6f84578efd92becabee7b2795f6dcffa479e6a"}
Jul 21 13:26:27.101557 ip-10-0-130-2 kubenswrapper[2556]: E0721 13:26:27.101523    2556 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7292-140084598090288\" is forbidden: User \"system:node:ip-10-0-130-2.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-130-2.ec2.internal' and this object" podUID="3f86c7f0-523e-4758-a349-6044f315c310" pod="opendatahub/test-curl-7292-140084598090288"
Jul 21 13:26:27.195270 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:27.195240    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3f86c7f0-523e-4758-a349-6044f315c310" path="/var/lib/kubelet/pods/3f86c7f0-523e-4758-a349-6044f315c310/volumes"
Jul 21 13:26:27.196297 ip-10-0-130-2 kubenswrapper[2556]: E0721 13:26:27.196274    2556 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7292-140084598090288\" is forbidden: User \"system:node:ip-10-0-130-2.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-130-2.ec2.internal' and this object" podUID="3f86c7f0-523e-4758-a349-6044f315c310" pod="opendatahub/test-curl-7292-140084598090288"
Jul 21 13:26:28.221755 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:28.221730    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7292-140084598023200"
Jul 21 13:26:28.244504 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:28.244479    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7292-140084598023200_7ad31adf-b6d9-4364-b7ea-a764f8274714/test-curl-7292-140084598023200/0.log"
Jul 21 13:26:28.272741 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:28.272718    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7292-140084598023200"]
Jul 21 13:26:28.274804 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:28.274786    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7292-140084598023200"]
Jul 21 13:26:28.350540 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:28.350479    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jqzmk\" (UniqueName: \"kubernetes.io/projected/7ad31adf-b6d9-4364-b7ea-a764f8274714-kube-api-access-jqzmk\") pod \"7ad31adf-b6d9-4364-b7ea-a764f8274714\" (UID: \"7ad31adf-b6d9-4364-b7ea-a764f8274714\") "
Jul 21 13:26:28.352623 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:28.352597    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7ad31adf-b6d9-4364-b7ea-a764f8274714-kube-api-access-jqzmk" (OuterVolumeSpecName: "kube-api-access-jqzmk") pod "7ad31adf-b6d9-4364-b7ea-a764f8274714" (UID: "7ad31adf-b6d9-4364-b7ea-a764f8274714"). InnerVolumeSpecName "kube-api-access-jqzmk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 13:26:28.397227 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:28.397202    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7292-140084598023200"]
Jul 21 13:26:28.397651 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:28.397637    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7ad31adf-b6d9-4364-b7ea-a764f8274714" containerName="test-curl-7292-140084598023200"
Jul 21 13:26:28.397697 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:28.397653    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="7ad31adf-b6d9-4364-b7ea-a764f8274714" containerName="test-curl-7292-140084598023200"
Jul 21 13:26:28.397747 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:28.397738    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="7ad31adf-b6d9-4364-b7ea-a764f8274714" containerName="test-curl-7292-140084598023200"
Jul 21 13:26:28.400722 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:28.400706    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7292-140084598023200"
Jul 21 13:26:28.407694 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:28.407674    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7292-140084598023200"]
Jul 21 13:26:28.451494 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:28.451466    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jqzmk\" (UniqueName: \"kubernetes.io/projected/7ad31adf-b6d9-4364-b7ea-a764f8274714-kube-api-access-jqzmk\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Jul 21 13:26:28.552754 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:28.552714    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jdlgv\" (UniqueName: \"kubernetes.io/projected/a97db358-00a1-4de5-88e7-9e7232ec040f-kube-api-access-jdlgv\") pod \"test-curl-7292-140084598023200\" (UID: \"a97db358-00a1-4de5-88e7-9e7232ec040f\") " pod="opendatahub/test-curl-7292-140084598023200"
Jul 21 13:26:28.654151 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:28.654087    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jdlgv\" (UniqueName: \"kubernetes.io/projected/a97db358-00a1-4de5-88e7-9e7232ec040f-kube-api-access-jdlgv\") pod \"test-curl-7292-140084598023200\" (UID: \"a97db358-00a1-4de5-88e7-9e7232ec040f\") " pod="opendatahub/test-curl-7292-140084598023200"
Jul 21 13:26:28.663646 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:28.663623    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jdlgv\" (UniqueName: \"kubernetes.io/projected/a97db358-00a1-4de5-88e7-9e7232ec040f-kube-api-access-jdlgv\") pod \"test-curl-7292-140084598023200\" (UID: \"a97db358-00a1-4de5-88e7-9e7232ec040f\") " pod="opendatahub/test-curl-7292-140084598023200"
Jul 21 13:26:28.712411 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:28.712389    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7292-140084598023200"
Jul 21 13:26:28.847786 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:28.847762    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7292-140084598023200"]
Jul 21 13:26:28.849961 ip-10-0-130-2 kubenswrapper[2556]: W0721 13:26:28.849936    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda97db358_00a1_4de5_88e7_9e7232ec040f.slice/crio-e79c352efcbd3b391ed8f54171df60ed464811f870f30fb9f14c9115fe065c3b WatchSource:0}: Error finding container e79c352efcbd3b391ed8f54171df60ed464811f870f30fb9f14c9115fe065c3b: Status 404 returned error can't find the container with id e79c352efcbd3b391ed8f54171df60ed464811f870f30fb9f14c9115fe065c3b
Jul 21 13:26:29.091056 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:29.090999    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7292-140084598023200"
Jul 21 13:26:29.091056 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:29.091043    2556 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="06058269e16f31ed6211e41b60b7d234296c2c4b14df8473b2f55d08a1d4c9b5"
Jul 21 13:26:29.092220 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:29.092198    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7292-140084598023200" event={"ID":"a97db358-00a1-4de5-88e7-9e7232ec040f","Type":"ContainerStarted","Data":"e79c352efcbd3b391ed8f54171df60ed464811f870f30fb9f14c9115fe065c3b"}
Jul 21 13:26:29.102746 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:29.102714    2556 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="opendatahub/test-curl-7292-140084598023200" oldPodUID="7ad31adf-b6d9-4364-b7ea-a764f8274714" podUID="a97db358-00a1-4de5-88e7-9e7232ec040f"
Jul 21 13:26:29.194425 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:29.194401    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7ad31adf-b6d9-4364-b7ea-a764f8274714" path="/var/lib/kubelet/pods/7ad31adf-b6d9-4364-b7ea-a764f8274714/volumes"
Jul 21 13:26:30.099557 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:30.099523    2556 generic.go:356] "Generic (PLEG): container finished" podID="a97db358-00a1-4de5-88e7-9e7232ec040f" containerID="bb3c372cd1d1b6de91409209900a8fb9c029dcb4f94d69228fe2cca0f7966ca9" exitCode=0
Jul 21 13:26:30.099960 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:30.099607    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7292-140084598023200" event={"ID":"a97db358-00a1-4de5-88e7-9e7232ec040f","Type":"ContainerDied","Data":"bb3c372cd1d1b6de91409209900a8fb9c029dcb4f94d69228fe2cca0f7966ca9"}
Jul 21 13:26:31.246129 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:31.246104    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7292-140084598023200"
Jul 21 13:26:31.268537 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:31.268514    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7292-140084598023200_a97db358-00a1-4de5-88e7-9e7232ec040f/test-curl-7292-140084598023200/0.log"
Jul 21 13:26:31.296749 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:31.296721    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7292-140084598023200"]
Jul 21 13:26:31.300889 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:31.300868    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7292-140084598023200"]
Jul 21 13:26:31.381864 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:31.381785    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jdlgv\" (UniqueName: \"kubernetes.io/projected/a97db358-00a1-4de5-88e7-9e7232ec040f-kube-api-access-jdlgv\") pod \"a97db358-00a1-4de5-88e7-9e7232ec040f\" (UID: \"a97db358-00a1-4de5-88e7-9e7232ec040f\") "
Jul 21 13:26:31.384107 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:31.384081    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a97db358-00a1-4de5-88e7-9e7232ec040f-kube-api-access-jdlgv" (OuterVolumeSpecName: "kube-api-access-jdlgv") pod "a97db358-00a1-4de5-88e7-9e7232ec040f" (UID: "a97db358-00a1-4de5-88e7-9e7232ec040f"). InnerVolumeSpecName "kube-api-access-jdlgv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 13:26:31.436667 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:31.436634    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7292-140084598024208"]
Jul 21 13:26:31.437086 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:31.437072    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a97db358-00a1-4de5-88e7-9e7232ec040f" containerName="test-curl-7292-140084598023200"
Jul 21 13:26:31.437086 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:31.437087    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="a97db358-00a1-4de5-88e7-9e7232ec040f" containerName="test-curl-7292-140084598023200"
Jul 21 13:26:31.437175 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:31.437165    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="a97db358-00a1-4de5-88e7-9e7232ec040f" containerName="test-curl-7292-140084598023200"
Jul 21 13:26:31.440199 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:31.440184    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7292-140084598024208"
Jul 21 13:26:31.458882 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:31.458809    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7292-140084598024208"]
Jul 21 13:26:31.482762 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:31.482733    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jdlgv\" (UniqueName: \"kubernetes.io/projected/a97db358-00a1-4de5-88e7-9e7232ec040f-kube-api-access-jdlgv\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Jul 21 13:26:31.583442 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:31.583404    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t5nfx\" (UniqueName: \"kubernetes.io/projected/4c781736-3d7a-4c1c-b83a-c559c38485c9-kube-api-access-t5nfx\") pod \"test-curl-7292-140084598024208\" (UID: \"4c781736-3d7a-4c1c-b83a-c559c38485c9\") " pod="opendatahub/test-curl-7292-140084598024208"
Jul 21 13:26:31.684442 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:31.684351    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t5nfx\" (UniqueName: \"kubernetes.io/projected/4c781736-3d7a-4c1c-b83a-c559c38485c9-kube-api-access-t5nfx\") pod \"test-curl-7292-140084598024208\" (UID: \"4c781736-3d7a-4c1c-b83a-c559c38485c9\") " pod="opendatahub/test-curl-7292-140084598024208"
Jul 21 13:26:31.694250 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:31.694223    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t5nfx\" (UniqueName: \"kubernetes.io/projected/4c781736-3d7a-4c1c-b83a-c559c38485c9-kube-api-access-t5nfx\") pod \"test-curl-7292-140084598024208\" (UID: \"4c781736-3d7a-4c1c-b83a-c559c38485c9\") " pod="opendatahub/test-curl-7292-140084598024208"
Jul 21 13:26:31.754863 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:31.754835    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7292-140084598024208"
Jul 21 13:26:31.903815 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:31.903785    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7292-140084598024208"]
Jul 21 13:26:31.904480 ip-10-0-130-2 kubenswrapper[2556]: W0721 13:26:31.904453    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4c781736_3d7a_4c1c_b83a_c559c38485c9.slice/crio-9d14c739d3277fa939404c010caa3ae6654ef5bd355f294b714a4c8eec22b579 WatchSource:0}: Error finding container 9d14c739d3277fa939404c010caa3ae6654ef5bd355f294b714a4c8eec22b579: Status 404 returned error can't find the container with id 9d14c739d3277fa939404c010caa3ae6654ef5bd355f294b714a4c8eec22b579
Jul 21 13:26:32.112029 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:32.111995    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7292-140084598023200"
Jul 21 13:26:32.112207 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:32.112062    2556 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e79c352efcbd3b391ed8f54171df60ed464811f870f30fb9f14c9115fe065c3b"
Jul 21 13:26:32.113309 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:32.113279    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7292-140084598024208" event={"ID":"4c781736-3d7a-4c1c-b83a-c559c38485c9","Type":"ContainerStarted","Data":"9d14c739d3277fa939404c010caa3ae6654ef5bd355f294b714a4c8eec22b579"}
Jul 21 13:26:32.122895 ip-10-0-130-2 kubenswrapper[2556]: E0721 13:26:32.122869    2556 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7292-140084598023200\" is forbidden: User \"system:node:ip-10-0-130-2.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-130-2.ec2.internal' and this object" podUID="a97db358-00a1-4de5-88e7-9e7232ec040f" pod="opendatahub/test-curl-7292-140084598023200"
Jul 21 13:26:32.771129 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:32.771079    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-cfed01/rate-test-model-cfed01-kserve-567c966cf5-df9ch" podUID="60c99958-b23f-4814-8db7-04c29b8e8d03" containerName="main" containerID="cri-o://b8d045b7dd28f4afafedc09a9a74d59b97e4a8b7daad4625596c78a81ebd1563" gracePeriod=45
Jul 21 13:26:33.040946 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:33.040922    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-cfed01/rate-test-model-cfed01-kserve-567c966cf5-df9ch"
Jul 21 13:26:33.070898 ip-10-0-130-2 kubenswrapper[2556]: E0721 13:26:33.070860    2556 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7292-140084598023200\" is forbidden: User \"system:node:ip-10-0-130-2.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-130-2.ec2.internal' and this object" podUID="a97db358-00a1-4de5-88e7-9e7232ec040f" pod="opendatahub/test-curl-7292-140084598023200"
Jul 21 13:26:33.120286 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:33.120256    2556 generic.go:356] "Generic (PLEG): container finished" podID="4c781736-3d7a-4c1c-b83a-c559c38485c9" containerID="f11dfccfb34a6ce28314b86846b1453a7844f8bb366d3647a7580a259b99dfda" exitCode=0
Jul 21 13:26:33.120491 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:33.120348    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7292-140084598024208" event={"ID":"4c781736-3d7a-4c1c-b83a-c559c38485c9","Type":"ContainerDied","Data":"f11dfccfb34a6ce28314b86846b1453a7844f8bb366d3647a7580a259b99dfda"}
Jul 21 13:26:33.121630 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:33.121611    2556 generic.go:356] "Generic (PLEG): container finished" podID="60c99958-b23f-4814-8db7-04c29b8e8d03" containerID="b8d045b7dd28f4afafedc09a9a74d59b97e4a8b7daad4625596c78a81ebd1563" exitCode=0
Jul 21 13:26:33.121728 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:33.121662    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-cfed01/rate-test-model-cfed01-kserve-567c966cf5-df9ch"
Jul 21 13:26:33.121728 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:33.121700    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-cfed01/rate-test-model-cfed01-kserve-567c966cf5-df9ch" event={"ID":"60c99958-b23f-4814-8db7-04c29b8e8d03","Type":"ContainerDied","Data":"b8d045b7dd28f4afafedc09a9a74d59b97e4a8b7daad4625596c78a81ebd1563"}
Jul 21 13:26:33.121853 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:33.121743    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-cfed01/rate-test-model-cfed01-kserve-567c966cf5-df9ch" event={"ID":"60c99958-b23f-4814-8db7-04c29b8e8d03","Type":"ContainerDied","Data":"6ef9ebbcb02398c92a5d088f072e3b434691fc6a50ba40b054747ad2d2c953f7"}
Jul 21 13:26:33.121853 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:33.121768    2556 scope.go:117] "RemoveContainer" containerID="b8d045b7dd28f4afafedc09a9a74d59b97e4a8b7daad4625596c78a81ebd1563"
Jul 21 13:26:33.130840 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:33.130825    2556 scope.go:117] "RemoveContainer" containerID="b8d045b7dd28f4afafedc09a9a74d59b97e4a8b7daad4625596c78a81ebd1563"
Jul 21 13:26:33.131129 ip-10-0-130-2 kubenswrapper[2556]: E0721 13:26:33.131113    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b8d045b7dd28f4afafedc09a9a74d59b97e4a8b7daad4625596c78a81ebd1563\": container with ID starting with b8d045b7dd28f4afafedc09a9a74d59b97e4a8b7daad4625596c78a81ebd1563 not found: ID does not exist" containerID="b8d045b7dd28f4afafedc09a9a74d59b97e4a8b7daad4625596c78a81ebd1563"
Jul 21 13:26:33.131181 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:33.131136    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b8d045b7dd28f4afafedc09a9a74d59b97e4a8b7daad4625596c78a81ebd1563"} err="failed to get container status \"b8d045b7dd28f4afafedc09a9a74d59b97e4a8b7daad4625596c78a81ebd1563\": rpc error: code = NotFound desc = could not find container \"b8d045b7dd28f4afafedc09a9a74d59b97e4a8b7daad4625596c78a81ebd1563\": container with ID starting with b8d045b7dd28f4afafedc09a9a74d59b97e4a8b7daad4625596c78a81ebd1563 not found: ID does not exist"
Jul 21 13:26:33.139427 ip-10-0-130-2 kubenswrapper[2556]: E0721 13:26:33.139398    2556 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7292-140084598023200\" is forbidden: User \"system:node:ip-10-0-130-2.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-130-2.ec2.internal' and this object" podUID="a97db358-00a1-4de5-88e7-9e7232ec040f" pod="opendatahub/test-curl-7292-140084598023200"
Jul 21 13:26:33.144193 ip-10-0-130-2 kubenswrapper[2556]: E0721 13:26:33.144171    2556 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7292-140084598023200\" is forbidden: User \"system:node:ip-10-0-130-2.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-130-2.ec2.internal' and this object" podUID="a97db358-00a1-4de5-88e7-9e7232ec040f" pod="opendatahub/test-curl-7292-140084598023200"
Jul 21 13:26:33.196501 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:33.196477    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a97db358-00a1-4de5-88e7-9e7232ec040f" path="/var/lib/kubelet/pods/a97db358-00a1-4de5-88e7-9e7232ec040f/volumes"
Jul 21 13:26:33.199441 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:33.199422    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/60c99958-b23f-4814-8db7-04c29b8e8d03-tls-certs\") pod \"60c99958-b23f-4814-8db7-04c29b8e8d03\" (UID: \"60c99958-b23f-4814-8db7-04c29b8e8d03\") "
Jul 21 13:26:33.199544 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:33.199450    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/60c99958-b23f-4814-8db7-04c29b8e8d03-model-cache\") pod \"60c99958-b23f-4814-8db7-04c29b8e8d03\" (UID: \"60c99958-b23f-4814-8db7-04c29b8e8d03\") "
Jul 21 13:26:33.199544 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:33.199478    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rd7lx\" (UniqueName: \"kubernetes.io/projected/60c99958-b23f-4814-8db7-04c29b8e8d03-kube-api-access-rd7lx\") pod \"60c99958-b23f-4814-8db7-04c29b8e8d03\" (UID: \"60c99958-b23f-4814-8db7-04c29b8e8d03\") "
Jul 21 13:26:33.199544 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:33.199506    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/60c99958-b23f-4814-8db7-04c29b8e8d03-dshm\") pod \"60c99958-b23f-4814-8db7-04c29b8e8d03\" (UID: \"60c99958-b23f-4814-8db7-04c29b8e8d03\") "
Jul 21 13:26:33.199714 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:33.199548    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/60c99958-b23f-4814-8db7-04c29b8e8d03-tmp-dir\") pod \"60c99958-b23f-4814-8db7-04c29b8e8d03\" (UID: \"60c99958-b23f-4814-8db7-04c29b8e8d03\") "
Jul 21 13:26:33.199714 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:33.199663    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/60c99958-b23f-4814-8db7-04c29b8e8d03-home\") pod \"60c99958-b23f-4814-8db7-04c29b8e8d03\" (UID: \"60c99958-b23f-4814-8db7-04c29b8e8d03\") "
Jul 21 13:26:33.199814 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:33.199730    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60c99958-b23f-4814-8db7-04c29b8e8d03-model-cache" (OuterVolumeSpecName: "model-cache") pod "60c99958-b23f-4814-8db7-04c29b8e8d03" (UID: "60c99958-b23f-4814-8db7-04c29b8e8d03"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 13:26:33.199927 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:33.199901    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60c99958-b23f-4814-8db7-04c29b8e8d03-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "60c99958-b23f-4814-8db7-04c29b8e8d03" (UID: "60c99958-b23f-4814-8db7-04c29b8e8d03"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 13:26:33.199989 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:33.199926    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60c99958-b23f-4814-8db7-04c29b8e8d03-home" (OuterVolumeSpecName: "home") pod "60c99958-b23f-4814-8db7-04c29b8e8d03" (UID: "60c99958-b23f-4814-8db7-04c29b8e8d03"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 13:26:33.200076 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:33.200037    2556 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/60c99958-b23f-4814-8db7-04c29b8e8d03-home\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Jul 21 13:26:33.200076 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:33.200054    2556 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/60c99958-b23f-4814-8db7-04c29b8e8d03-model-cache\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Jul 21 13:26:33.200076 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:33.200066    2556 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/60c99958-b23f-4814-8db7-04c29b8e8d03-tmp-dir\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Jul 21 13:26:33.201858 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:33.201834    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60c99958-b23f-4814-8db7-04c29b8e8d03-dshm" (OuterVolumeSpecName: "dshm") pod "60c99958-b23f-4814-8db7-04c29b8e8d03" (UID: "60c99958-b23f-4814-8db7-04c29b8e8d03"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 13:26:33.201953 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:33.201834    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/60c99958-b23f-4814-8db7-04c29b8e8d03-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "60c99958-b23f-4814-8db7-04c29b8e8d03" (UID: "60c99958-b23f-4814-8db7-04c29b8e8d03"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 13:26:33.201953 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:33.201865    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/60c99958-b23f-4814-8db7-04c29b8e8d03-kube-api-access-rd7lx" (OuterVolumeSpecName: "kube-api-access-rd7lx") pod "60c99958-b23f-4814-8db7-04c29b8e8d03" (UID: "60c99958-b23f-4814-8db7-04c29b8e8d03"). InnerVolumeSpecName "kube-api-access-rd7lx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 13:26:33.301461 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:33.301392    2556 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/60c99958-b23f-4814-8db7-04c29b8e8d03-tls-certs\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Jul 21 13:26:33.301461 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:33.301417    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rd7lx\" (UniqueName: \"kubernetes.io/projected/60c99958-b23f-4814-8db7-04c29b8e8d03-kube-api-access-rd7lx\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Jul 21 13:26:33.301461 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:33.301427    2556 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/60c99958-b23f-4814-8db7-04c29b8e8d03-dshm\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Jul 21 13:26:33.446822 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:33.446795    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-cfed01/rate-test-model-cfed01-kserve-567c966cf5-df9ch"]
Jul 21 13:26:33.451983 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:33.451960    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-cfed01/rate-test-model-cfed01-kserve-567c966cf5-df9ch"]
Jul 21 13:26:34.273102 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:34.273082    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7292-140084598024208"
Jul 21 13:26:34.295567 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:34.295543    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7292-140084598024208_4c781736-3d7a-4c1c-b83a-c559c38485c9/test-curl-7292-140084598024208/0.log"
Jul 21 13:26:34.309506 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:34.309484    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t5nfx\" (UniqueName: \"kubernetes.io/projected/4c781736-3d7a-4c1c-b83a-c559c38485c9-kube-api-access-t5nfx\") pod \"4c781736-3d7a-4c1c-b83a-c559c38485c9\" (UID: \"4c781736-3d7a-4c1c-b83a-c559c38485c9\") "
Jul 21 13:26:34.311719 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:34.311692    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4c781736-3d7a-4c1c-b83a-c559c38485c9-kube-api-access-t5nfx" (OuterVolumeSpecName: "kube-api-access-t5nfx") pod "4c781736-3d7a-4c1c-b83a-c559c38485c9" (UID: "4c781736-3d7a-4c1c-b83a-c559c38485c9"). InnerVolumeSpecName "kube-api-access-t5nfx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 13:26:34.325201 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:34.325178    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7292-140084598024208"]
Jul 21 13:26:34.328588 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:34.328566    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7292-140084598024208"]
Jul 21 13:26:34.410829 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:34.410763    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t5nfx\" (UniqueName: \"kubernetes.io/projected/4c781736-3d7a-4c1c-b83a-c559c38485c9-kube-api-access-t5nfx\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Jul 21 13:26:34.446530 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:34.446505    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7292-140084598024208"]
Jul 21 13:26:34.446959 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:34.446947    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="60c99958-b23f-4814-8db7-04c29b8e8d03" containerName="main"
Jul 21 13:26:34.447003 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:34.446961    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="60c99958-b23f-4814-8db7-04c29b8e8d03" containerName="main"
Jul 21 13:26:34.447003 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:34.446975    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4c781736-3d7a-4c1c-b83a-c559c38485c9" containerName="test-curl-7292-140084598024208"
Jul 21 13:26:34.447003 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:34.446982    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="4c781736-3d7a-4c1c-b83a-c559c38485c9" containerName="test-curl-7292-140084598024208"
Jul 21 13:26:34.447118 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:34.447076    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="4c781736-3d7a-4c1c-b83a-c559c38485c9" containerName="test-curl-7292-140084598024208"
Jul 21 13:26:34.447118 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:34.447085    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="60c99958-b23f-4814-8db7-04c29b8e8d03" containerName="main"
Jul 21 13:26:34.450133 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:34.450117    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7292-140084598024208"
Jul 21 13:26:34.457606 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:34.457529    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7292-140084598024208"]
Jul 21 13:26:34.511400 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:34.511375    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7r9z2\" (UniqueName: \"kubernetes.io/projected/4e0df6db-000e-4b53-8f77-4500b68a5ddb-kube-api-access-7r9z2\") pod \"test-curl-7292-140084598024208\" (UID: \"4e0df6db-000e-4b53-8f77-4500b68a5ddb\") " pod="opendatahub/test-curl-7292-140084598024208"
Jul 21 13:26:34.612090 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:34.612055    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7r9z2\" (UniqueName: \"kubernetes.io/projected/4e0df6db-000e-4b53-8f77-4500b68a5ddb-kube-api-access-7r9z2\") pod \"test-curl-7292-140084598024208\" (UID: \"4e0df6db-000e-4b53-8f77-4500b68a5ddb\") " pod="opendatahub/test-curl-7292-140084598024208"
Jul 21 13:26:34.621679 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:34.621649    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7r9z2\" (UniqueName: \"kubernetes.io/projected/4e0df6db-000e-4b53-8f77-4500b68a5ddb-kube-api-access-7r9z2\") pod \"test-curl-7292-140084598024208\" (UID: \"4e0df6db-000e-4b53-8f77-4500b68a5ddb\") " pod="opendatahub/test-curl-7292-140084598024208"
Jul 21 13:26:34.759736 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:34.759701    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7292-140084598024208"
Jul 21 13:26:34.900168 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:34.900140    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7292-140084598024208"]
Jul 21 13:26:34.901925 ip-10-0-130-2 kubenswrapper[2556]: W0721 13:26:34.901900    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4e0df6db_000e_4b53_8f77_4500b68a5ddb.slice/crio-5c44cd9d85b08bff92387cc75bfc26a72946b27ba1bc361fb42f1e99b3da97eb WatchSource:0}: Error finding container 5c44cd9d85b08bff92387cc75bfc26a72946b27ba1bc361fb42f1e99b3da97eb: Status 404 returned error can't find the container with id 5c44cd9d85b08bff92387cc75bfc26a72946b27ba1bc361fb42f1e99b3da97eb
Jul 21 13:26:35.135481 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:35.135397    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7292-140084598024208"
Jul 21 13:26:35.135481 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:35.135412    2556 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9d14c739d3277fa939404c010caa3ae6654ef5bd355f294b714a4c8eec22b579"
Jul 21 13:26:35.136620 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:35.136596    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7292-140084598024208" event={"ID":"4e0df6db-000e-4b53-8f77-4500b68a5ddb","Type":"ContainerStarted","Data":"5c44cd9d85b08bff92387cc75bfc26a72946b27ba1bc361fb42f1e99b3da97eb"}
Jul 21 13:26:35.147525 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:35.147501    2556 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="opendatahub/test-curl-7292-140084598024208" oldPodUID="4c781736-3d7a-4c1c-b83a-c559c38485c9" podUID="4e0df6db-000e-4b53-8f77-4500b68a5ddb"
Jul 21 13:26:35.197932 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:35.197902    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4c781736-3d7a-4c1c-b83a-c559c38485c9" path="/var/lib/kubelet/pods/4c781736-3d7a-4c1c-b83a-c559c38485c9/volumes"
Jul 21 13:26:35.198340 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:35.198324    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="60c99958-b23f-4814-8db7-04c29b8e8d03" path="/var/lib/kubelet/pods/60c99958-b23f-4814-8db7-04c29b8e8d03/volumes"
Jul 21 13:26:36.142925 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:36.142887    2556 generic.go:356] "Generic (PLEG): container finished" podID="4e0df6db-000e-4b53-8f77-4500b68a5ddb" containerID="028dde32ca76cef05459e448c8b1a7cfcb1b89dc30f3c82680a62c1361fab0df" exitCode=0
Jul 21 13:26:36.143330 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:36.142975    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7292-140084598024208" event={"ID":"4e0df6db-000e-4b53-8f77-4500b68a5ddb","Type":"ContainerDied","Data":"028dde32ca76cef05459e448c8b1a7cfcb1b89dc30f3c82680a62c1361fab0df"}
Jul 21 13:26:37.300538 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:37.300516    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7292-140084598024208"
Jul 21 13:26:37.331703 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:37.331666    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7292-140084598024208_4e0df6db-000e-4b53-8f77-4500b68a5ddb/test-curl-7292-140084598024208/0.log"
Jul 21 13:26:37.339514 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:37.339488    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7r9z2\" (UniqueName: \"kubernetes.io/projected/4e0df6db-000e-4b53-8f77-4500b68a5ddb-kube-api-access-7r9z2\") pod \"4e0df6db-000e-4b53-8f77-4500b68a5ddb\" (UID: \"4e0df6db-000e-4b53-8f77-4500b68a5ddb\") "
Jul 21 13:26:37.341780 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:37.341750    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4e0df6db-000e-4b53-8f77-4500b68a5ddb-kube-api-access-7r9z2" (OuterVolumeSpecName: "kube-api-access-7r9z2") pod "4e0df6db-000e-4b53-8f77-4500b68a5ddb" (UID: "4e0df6db-000e-4b53-8f77-4500b68a5ddb"). InnerVolumeSpecName "kube-api-access-7r9z2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 13:26:37.367303 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:37.367278    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7292-140084598024208"]
Jul 21 13:26:37.371812 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:37.371792    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7292-140084598024208"]
Jul 21 13:26:37.441000 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:37.440939    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7r9z2\" (UniqueName: \"kubernetes.io/projected/4e0df6db-000e-4b53-8f77-4500b68a5ddb-kube-api-access-7r9z2\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Jul 21 13:26:37.568760 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:37.568732    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-b3a013-x9z8v"]
Jul 21 13:26:37.569188 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:37.569174    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e0df6db-000e-4b53-8f77-4500b68a5ddb" containerName="test-curl-7292-140084598024208"
Jul 21 13:26:37.569245 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:37.569189    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e0df6db-000e-4b53-8f77-4500b68a5ddb" containerName="test-curl-7292-140084598024208"
Jul 21 13:26:37.569299 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:37.569288    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="4e0df6db-000e-4b53-8f77-4500b68a5ddb" containerName="test-curl-7292-140084598024208"
Jul 21 13:26:37.572365 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:37.572349    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-b3a013-x9z8v"
Jul 21 13:26:37.574747 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:37.574730    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5nsrr\""
Jul 21 13:26:37.583107 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:37.583084    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-b3a013-x9z8v"]
Jul 21 13:26:37.643225 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:37.643199    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c7592a3d-7262-47d1-9e30-7eb0d8ff4340-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-b3a013-x9z8v\" (UID: \"c7592a3d-7262-47d1-9e30-7eb0d8ff4340\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-b3a013-x9z8v"
Jul 21 13:26:37.744319 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:37.744291    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c7592a3d-7262-47d1-9e30-7eb0d8ff4340-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-b3a013-x9z8v\" (UID: \"c7592a3d-7262-47d1-9e30-7eb0d8ff4340\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-b3a013-x9z8v"
Jul 21 13:26:37.744891 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:37.744872    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c7592a3d-7262-47d1-9e30-7eb0d8ff4340-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-b3a013-x9z8v\" (UID: \"c7592a3d-7262-47d1-9e30-7eb0d8ff4340\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-b3a013-x9z8v"
Jul 21 13:26:37.884113 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:37.884085    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-b3a013-x9z8v"
Jul 21 13:26:38.033479 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:38.033455    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-b3a013-x9z8v"]
Jul 21 13:26:38.035435 ip-10-0-130-2 kubenswrapper[2556]: W0721 13:26:38.035412    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc7592a3d_7262_47d1_9e30_7eb0d8ff4340.slice/crio-c603744dd42de884c2c8a0d42eee0160691ef6d203f8ef00a93c426ccc40e604 WatchSource:0}: Error finding container c603744dd42de884c2c8a0d42eee0160691ef6d203f8ef00a93c426ccc40e604: Status 404 returned error can't find the container with id c603744dd42de884c2c8a0d42eee0160691ef6d203f8ef00a93c426ccc40e604
Jul 21 13:26:38.155273 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:38.155251    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7292-140084598024208"
Jul 21 13:26:38.155404 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:38.155261    2556 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5c44cd9d85b08bff92387cc75bfc26a72946b27ba1bc361fb42f1e99b3da97eb"
Jul 21 13:26:38.156542 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:38.156521    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-b3a013-x9z8v" event={"ID":"c7592a3d-7262-47d1-9e30-7eb0d8ff4340","Type":"ContainerStarted","Data":"c603744dd42de884c2c8a0d42eee0160691ef6d203f8ef00a93c426ccc40e604"}
Jul 21 13:26:38.166138 ip-10-0-130-2 kubenswrapper[2556]: E0721 13:26:38.166108    2556 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7292-140084598024208\" is forbidden: User \"system:node:ip-10-0-130-2.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-130-2.ec2.internal' and this object" podUID="4e0df6db-000e-4b53-8f77-4500b68a5ddb" pod="opendatahub/test-curl-7292-140084598024208"
Jul 21 13:26:39.164573 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:39.164535    2556 generic.go:356] "Generic (PLEG): container finished" podID="c7592a3d-7262-47d1-9e30-7eb0d8ff4340" containerID="8e1a97d71fdda41f819345acd7660ecaf6c438655197b21c62ce000a7bb5c751" exitCode=0
Jul 21 13:26:39.164972 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:39.164593    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-b3a013-x9z8v" event={"ID":"c7592a3d-7262-47d1-9e30-7eb0d8ff4340","Type":"ContainerDied","Data":"8e1a97d71fdda41f819345acd7660ecaf6c438655197b21c62ce000a7bb5c751"}
Jul 21 13:26:39.182925 ip-10-0-130-2 kubenswrapper[2556]: E0721 13:26:39.182885    2556 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7292-140084598024208\" is forbidden: User \"system:node:ip-10-0-130-2.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-130-2.ec2.internal' and this object" podUID="4e0df6db-000e-4b53-8f77-4500b68a5ddb" pod="opendatahub/test-curl-7292-140084598024208"
Jul 21 13:26:39.195472 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:39.195449    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4e0df6db-000e-4b53-8f77-4500b68a5ddb" path="/var/lib/kubelet/pods/4e0df6db-000e-4b53-8f77-4500b68a5ddb/volumes"
Jul 21 13:26:40.319584 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:40.319556    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-b3a013-x9z8v"
Jul 21 13:26:40.369636 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:40.369606    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c7592a3d-7262-47d1-9e30-7eb0d8ff4340-maas-api-service-ca\") pod \"c7592a3d-7262-47d1-9e30-7eb0d8ff4340\" (UID: \"c7592a3d-7262-47d1-9e30-7eb0d8ff4340\") "
Jul 21 13:26:40.369982 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:40.369945    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c7592a3d-7262-47d1-9e30-7eb0d8ff4340-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "c7592a3d-7262-47d1-9e30-7eb0d8ff4340" (UID: "c7592a3d-7262-47d1-9e30-7eb0d8ff4340"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 13:26:40.470582 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:40.470495    2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c7592a3d-7262-47d1-9e30-7eb0d8ff4340-maas-api-service-ca\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Jul 21 13:26:41.177482 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:41.177449    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-b3a013-x9z8v" event={"ID":"c7592a3d-7262-47d1-9e30-7eb0d8ff4340","Type":"ContainerDied","Data":"c603744dd42de884c2c8a0d42eee0160691ef6d203f8ef00a93c426ccc40e604"}
Jul 21 13:26:41.177482 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:41.177484    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-b3a013-x9z8v"
Jul 21 13:26:41.177690 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:41.177527    2556 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c603744dd42de884c2c8a0d42eee0160691ef6d203f8ef00a93c426ccc40e604"
Jul 21 13:26:47.600769 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:47.600738    2556 scope.go:117] "RemoveContainer" containerID="8ed2be328c57511ce5d858835f042bf11d1338a6cd3c346fe6e00360be48fee3"
Jul 21 13:26:52.690669 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:52.690635    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-b3a013-x9z8v"]
Jul 21 13:26:52.696217 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:52.696192    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-b3a013-x9z8v"]
Jul 21 13:26:53.195288 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:26:53.195253    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c7592a3d-7262-47d1-9e30-7eb0d8ff4340" path="/var/lib/kubelet/pods/c7592a3d-7262-47d1-9e30-7eb0d8ff4340/volumes"
Jul 21 13:27:00.456815 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:27:00.456782    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-b3a013-openshift-default-5648c9c987-nzzwb"]
Jul 21 13:27:00.457311 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:27:00.457061    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-b3a013-openshift-default-5648c9c987-nzzwb" podUID="b0034705-9d87-44f2-b191-aabfaabfeb6e" containerName="istio-proxy" containerID="cri-o://5b3e09930c2ac4a297d0e4c63813997c546a76c6a896ddc48a549528af8087a2" gracePeriod=30
Jul 21 13:27:04.308780 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:27:04.308747    2556 patch_prober.go:28] interesting pod/e2e-shared-a-b3a013-openshift-default-5648c9c987-nzzwb container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 21 13:27:04.309232 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:27:04.308808    2556 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-b3a013-openshift-default-5648c9c987-nzzwb" podUID="b0034705-9d87-44f2-b191-aabfaabfeb6e" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 21 13:27:05.714889 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:27:05.714858    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-b3a013-openshift-default-5648c9c987-nzzwb"
Jul 21 13:27:05.803617 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:27:05.803595    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b0034705-9d87-44f2-b191-aabfaabfeb6e-istio-data\") pod \"b0034705-9d87-44f2-b191-aabfaabfeb6e\" (UID: \"b0034705-9d87-44f2-b191-aabfaabfeb6e\") "
Jul 21 13:27:05.803762 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:27:05.803650    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b0034705-9d87-44f2-b191-aabfaabfeb6e-istio-podinfo\") pod \"b0034705-9d87-44f2-b191-aabfaabfeb6e\" (UID: \"b0034705-9d87-44f2-b191-aabfaabfeb6e\") "
Jul 21 13:27:05.803762 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:27:05.803667    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b0034705-9d87-44f2-b191-aabfaabfeb6e-istio-token\") pod \"b0034705-9d87-44f2-b191-aabfaabfeb6e\" (UID: \"b0034705-9d87-44f2-b191-aabfaabfeb6e\") "
Jul 21 13:27:05.803762 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:27:05.803698    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b0034705-9d87-44f2-b191-aabfaabfeb6e-workload-certs\") pod \"b0034705-9d87-44f2-b191-aabfaabfeb6e\" (UID: \"b0034705-9d87-44f2-b191-aabfaabfeb6e\") "
Jul 21 13:27:05.803762 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:27:05.803719    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0034705-9d87-44f2-b191-aabfaabfeb6e-credential-socket\") pod \"b0034705-9d87-44f2-b191-aabfaabfeb6e\" (UID: \"b0034705-9d87-44f2-b191-aabfaabfeb6e\") "
Jul 21 13:27:05.803762 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:27:05.803741    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0034705-9d87-44f2-b191-aabfaabfeb6e-workload-socket\") pod \"b0034705-9d87-44f2-b191-aabfaabfeb6e\" (UID: \"b0034705-9d87-44f2-b191-aabfaabfeb6e\") "
Jul 21 13:27:05.804086 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:27:05.803799    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b0034705-9d87-44f2-b191-aabfaabfeb6e-istio-envoy\") pod \"b0034705-9d87-44f2-b191-aabfaabfeb6e\" (UID: \"b0034705-9d87-44f2-b191-aabfaabfeb6e\") "
Jul 21 13:27:05.804086 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:27:05.803846    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b0034705-9d87-44f2-b191-aabfaabfeb6e-istiod-ca-cert\") pod \"b0034705-9d87-44f2-b191-aabfaabfeb6e\" (UID: \"b0034705-9d87-44f2-b191-aabfaabfeb6e\") "
Jul 21 13:27:05.804086 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:27:05.803897    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9894l\" (UniqueName: \"kubernetes.io/projected/b0034705-9d87-44f2-b191-aabfaabfeb6e-kube-api-access-9894l\") pod \"b0034705-9d87-44f2-b191-aabfaabfeb6e\" (UID: \"b0034705-9d87-44f2-b191-aabfaabfeb6e\") "
Jul 21 13:27:05.804086 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:27:05.804046    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0034705-9d87-44f2-b191-aabfaabfeb6e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b0034705-9d87-44f2-b191-aabfaabfeb6e" (UID: "b0034705-9d87-44f2-b191-aabfaabfeb6e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 13:27:05.804345 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:27:05.804076    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0034705-9d87-44f2-b191-aabfaabfeb6e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b0034705-9d87-44f2-b191-aabfaabfeb6e" (UID: "b0034705-9d87-44f2-b191-aabfaabfeb6e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 13:27:05.804345 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:27:05.804258    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0034705-9d87-44f2-b191-aabfaabfeb6e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b0034705-9d87-44f2-b191-aabfaabfeb6e" (UID: "b0034705-9d87-44f2-b191-aabfaabfeb6e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 13:27:05.804345 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:27:05.804272    2556 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b0034705-9d87-44f2-b191-aabfaabfeb6e-workload-certs\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Jul 21 13:27:05.804345 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:27:05.804290    2556 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0034705-9d87-44f2-b191-aabfaabfeb6e-credential-socket\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Jul 21 13:27:05.804345 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:27:05.804338    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b0034705-9d87-44f2-b191-aabfaabfeb6e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b0034705-9d87-44f2-b191-aabfaabfeb6e" (UID: "b0034705-9d87-44f2-b191-aabfaabfeb6e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 13:27:05.804576 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:27:05.804502    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0034705-9d87-44f2-b191-aabfaabfeb6e-istio-data" (OuterVolumeSpecName: "istio-data") pod "b0034705-9d87-44f2-b191-aabfaabfeb6e" (UID: "b0034705-9d87-44f2-b191-aabfaabfeb6e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 13:27:05.805953 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:27:05.805928    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b0034705-9d87-44f2-b191-aabfaabfeb6e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b0034705-9d87-44f2-b191-aabfaabfeb6e" (UID: "b0034705-9d87-44f2-b191-aabfaabfeb6e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 21 13:27:05.806148 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:27:05.806005    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b0034705-9d87-44f2-b191-aabfaabfeb6e-istio-token" (OuterVolumeSpecName: "istio-token") pod "b0034705-9d87-44f2-b191-aabfaabfeb6e" (UID: "b0034705-9d87-44f2-b191-aabfaabfeb6e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 13:27:05.806311 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:27:05.806293    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b0034705-9d87-44f2-b191-aabfaabfeb6e-kube-api-access-9894l" (OuterVolumeSpecName: "kube-api-access-9894l") pod "b0034705-9d87-44f2-b191-aabfaabfeb6e" (UID: "b0034705-9d87-44f2-b191-aabfaabfeb6e"). InnerVolumeSpecName "kube-api-access-9894l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 13:27:05.806370 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:27:05.806349    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0034705-9d87-44f2-b191-aabfaabfeb6e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b0034705-9d87-44f2-b191-aabfaabfeb6e" (UID: "b0034705-9d87-44f2-b191-aabfaabfeb6e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 13:27:05.905188 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:27:05.905161    2556 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b0034705-9d87-44f2-b191-aabfaabfeb6e-istio-envoy\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Jul 21 13:27:05.905188 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:27:05.905185    2556 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b0034705-9d87-44f2-b191-aabfaabfeb6e-istiod-ca-cert\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Jul 21 13:27:05.905338 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:27:05.905199    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9894l\" (UniqueName: \"kubernetes.io/projected/b0034705-9d87-44f2-b191-aabfaabfeb6e-kube-api-access-9894l\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Jul 21 13:27:05.905338 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:27:05.905213    2556 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b0034705-9d87-44f2-b191-aabfaabfeb6e-istio-data\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Jul 21 13:27:05.905338 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:27:05.905225    2556 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b0034705-9d87-44f2-b191-aabfaabfeb6e-istio-podinfo\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Jul 21 13:27:05.905338 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:27:05.905235    2556 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b0034705-9d87-44f2-b191-aabfaabfeb6e-istio-token\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Jul 21 13:27:05.905338 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:27:05.905248    2556 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0034705-9d87-44f2-b191-aabfaabfeb6e-workload-socket\") on node \"ip-10-0-130-2.ec2.internal\" DevicePath \"\""
Jul 21 13:27:06.314737 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:27:06.314709    2556 generic.go:356] "Generic (PLEG): container finished" podID="b0034705-9d87-44f2-b191-aabfaabfeb6e" containerID="5b3e09930c2ac4a297d0e4c63813997c546a76c6a896ddc48a549528af8087a2" exitCode=0
Jul 21 13:27:06.314883 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:27:06.314766    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-b3a013-openshift-default-5648c9c987-nzzwb"
Jul 21 13:27:06.314883 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:27:06.314805    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-b3a013-openshift-default-5648c9c987-nzzwb" event={"ID":"b0034705-9d87-44f2-b191-aabfaabfeb6e","Type":"ContainerDied","Data":"5b3e09930c2ac4a297d0e4c63813997c546a76c6a896ddc48a549528af8087a2"}
Jul 21 13:27:06.314883 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:27:06.314877    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-b3a013-openshift-default-5648c9c987-nzzwb" event={"ID":"b0034705-9d87-44f2-b191-aabfaabfeb6e","Type":"ContainerDied","Data":"ac8995b5bdd969acfab3130141e715a7a51f0aaff5705c4b1b4cc245668d3405"}
Jul 21 13:27:06.315080 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:27:06.314909    2556 scope.go:117] "RemoveContainer" containerID="5b3e09930c2ac4a297d0e4c63813997c546a76c6a896ddc48a549528af8087a2"
Jul 21 13:27:06.325111 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:27:06.325093    2556 scope.go:117] "RemoveContainer" containerID="0790c499256d1e026d0294b67a8c773d5abbcfd550b32d3a4dcb98799c469a64"
Jul 21 13:27:06.334097 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:27:06.334075    2556 scope.go:117] "RemoveContainer" containerID="5b3e09930c2ac4a297d0e4c63813997c546a76c6a896ddc48a549528af8087a2"
Jul 21 13:27:06.334354 ip-10-0-130-2 kubenswrapper[2556]: E0721 13:27:06.334337    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5b3e09930c2ac4a297d0e4c63813997c546a76c6a896ddc48a549528af8087a2\": container with ID starting with 5b3e09930c2ac4a297d0e4c63813997c546a76c6a896ddc48a549528af8087a2 not found: ID does not exist" containerID="5b3e09930c2ac4a297d0e4c63813997c546a76c6a896ddc48a549528af8087a2"
Jul 21 13:27:06.334407 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:27:06.334361    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5b3e09930c2ac4a297d0e4c63813997c546a76c6a896ddc48a549528af8087a2"} err="failed to get container status \"5b3e09930c2ac4a297d0e4c63813997c546a76c6a896ddc48a549528af8087a2\": rpc error: code = NotFound desc = could not find container \"5b3e09930c2ac4a297d0e4c63813997c546a76c6a896ddc48a549528af8087a2\": container with ID starting with 5b3e09930c2ac4a297d0e4c63813997c546a76c6a896ddc48a549528af8087a2 not found: ID does not exist"
Jul 21 13:27:06.334407 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:27:06.334378    2556 scope.go:117] "RemoveContainer" containerID="0790c499256d1e026d0294b67a8c773d5abbcfd550b32d3a4dcb98799c469a64"
Jul 21 13:27:06.334657 ip-10-0-130-2 kubenswrapper[2556]: E0721 13:27:06.334636    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0790c499256d1e026d0294b67a8c773d5abbcfd550b32d3a4dcb98799c469a64\": container with ID starting with 0790c499256d1e026d0294b67a8c773d5abbcfd550b32d3a4dcb98799c469a64 not found: ID does not exist" containerID="0790c499256d1e026d0294b67a8c773d5abbcfd550b32d3a4dcb98799c469a64"
Jul 21 13:27:06.334713 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:27:06.334665    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0790c499256d1e026d0294b67a8c773d5abbcfd550b32d3a4dcb98799c469a64"} err="failed to get container status \"0790c499256d1e026d0294b67a8c773d5abbcfd550b32d3a4dcb98799c469a64\": rpc error: code = NotFound desc = could not find container \"0790c499256d1e026d0294b67a8c773d5abbcfd550b32d3a4dcb98799c469a64\": container with ID starting with 0790c499256d1e026d0294b67a8c773d5abbcfd550b32d3a4dcb98799c469a64 not found: ID does not exist"
Jul 21 13:27:06.338785 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:27:06.338761    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-b3a013-openshift-default-5648c9c987-nzzwb"]
Jul 21 13:27:06.342957 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:27:06.342938    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-b3a013-openshift-default-5648c9c987-nzzwb"]
Jul 21 13:27:07.195521 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:27:07.195486    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b0034705-9d87-44f2-b191-aabfaabfeb6e" path="/var/lib/kubelet/pods/b0034705-9d87-44f2-b191-aabfaabfeb6e/volumes"
Jul 21 13:27:29.552368 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:27:29.552336    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-6r5kh_68425978-0369-4748-ba6f-b3cac04a2552/manager/0.log"
Jul 21 13:27:29.682666 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:27:29.682634    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-94vcn_7c34ea1b-966c-490f-9cbb-f9ff00bc95d8/manager/0.log"
Jul 21 13:27:29.954098 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:27:29.954009    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-phbht_a9b7bd6c-d5fa-4c79-890e-7509ba41dd4f/server/0.log"
Jul 21 13:27:30.077097 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:27:30.077075    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-s7tkd_7bc3e182-acec-4866-9dd0-f3db1e521186/manager/0.log"
Jul 21 13:27:30.505237 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:27:30.505213    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-579d89bddd-trnmc_226755f7-2596-454a-a055-36be3e543967/copy-manifests/0.log"
Jul 21 13:27:30.513875 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:27:30.513834    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-579d89bddd-trnmc_226755f7-2596-454a-a055-36be3e543967/manager/0.log"
Jul 21 13:27:30.889961 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:27:30.889893    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-cf4f9bc89-zsldg_621e7dbe-d55a-4e01-94e7-a30afdf644d6/maas-api/0.log"
Jul 21 13:27:31.006249 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:27:31.006227    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29743980-dk8kb_99fdf9c5-69d0-443c-ae68-04003cd37a46/cleanup/0.log"
Jul 21 13:27:31.241282 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:27:31.235763    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-44bvs_a9693639-fb3b-495e-9669-bf8a0a2b7127/postgres/0.log"
Jul 21 13:27:32.770417 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:27:32.770387    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-pmr2h_7962da2c-145f-499e-82d3-be635b223b41/manager/0.log"
Jul 21 13:27:33.156484 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:27:33.156415    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-k5kss_b134efa5-61c2-46f9-bc3e-50cc5dfc5b61/registry-server/0.log"
Jul 21 13:27:33.554511 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:27:33.554484    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-xj2dd_ad723ec0-e63c-4a5e-acda-719533071ba2/manager/0.log"
Jul 21 13:27:33.923554 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:27:33.923485    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4gldrd_856c41ff-2032-4fbd-91a2-44a28c8682ce/istio-proxy/0.log"
Jul 21 13:27:34.294645 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:27:34.294622    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-67cb6d6b5d-rfwlt_041ed91e-0571-4917-a641-c554b8734229/kube-auth-proxy/0.log"
Jul 21 13:27:34.420588 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:27:34.420565    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-b55kr_fe4ed9c9-5046-4bb2-acbd-8d83d8b8eb2c/istio-proxy/1.log"
Jul 21 13:27:34.555296 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:27:34.555237    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-xmqdx_ed44480e-4c2e-4539-8f87-d6af31005e3d/payload-pre-processing/0.log"
Jul 21 13:27:34.681642 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:27:34.681616    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-q5zgh_0e9d6e83-1286-4634-866a-3ffcdcb84dbc/payload-processing/0.log"
Jul 21 13:27:35.304263 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:27:35.304237    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-ng878_067dd5c2-04fa-471c-bdd3-652ea5c57333/main/0.log"
Jul 21 13:27:35.670672 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:27:35.670598    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-9skqr_bb3d9bf3-7469-41d0-bc43-1e251e23618d/main/0.log"
Jul 21 13:27:42.089719 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:27:42.089684    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-s9mdb/must-gather-vbnvb"]
Jul 21 13:27:42.090193 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:27:42.090173    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b0034705-9d87-44f2-b191-aabfaabfeb6e" containerName="istio-proxy"
Jul 21 13:27:42.090193 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:27:42.090188    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="b0034705-9d87-44f2-b191-aabfaabfeb6e" containerName="istio-proxy"
Jul 21 13:27:42.090193 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:27:42.090197    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b0034705-9d87-44f2-b191-aabfaabfeb6e" containerName="istio-proxy"
Jul 21 13:27:42.090321 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:27:42.090203    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="b0034705-9d87-44f2-b191-aabfaabfeb6e" containerName="istio-proxy"
Jul 21 13:27:42.090321 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:27:42.090218    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c7592a3d-7262-47d1-9e30-7eb0d8ff4340" containerName="revoke-keys"
Jul 21 13:27:42.090321 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:27:42.090225    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="c7592a3d-7262-47d1-9e30-7eb0d8ff4340" containerName="revoke-keys"
Jul 21 13:27:42.090321 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:27:42.090294    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="b0034705-9d87-44f2-b191-aabfaabfeb6e" containerName="istio-proxy"
Jul 21 13:27:42.090321 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:27:42.090303    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="c7592a3d-7262-47d1-9e30-7eb0d8ff4340" containerName="revoke-keys"
Jul 21 13:27:42.090495 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:27:42.090432    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="b0034705-9d87-44f2-b191-aabfaabfeb6e" containerName="istio-proxy"
Jul 21 13:27:42.093644 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:27:42.093625    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-s9mdb/must-gather-vbnvb"
Jul 21 13:27:42.095890 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:27:42.095867    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-s9mdb\"/\"default-dockercfg-kpxh2\""
Jul 21 13:27:42.096590 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:27:42.096573    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-s9mdb\"/\"kube-root-ca.crt\""
Jul 21 13:27:42.096680 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:27:42.096639    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-s9mdb\"/\"openshift-service-ca.crt\""
Jul 21 13:27:42.102112 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:27:42.102044    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-s9mdb/must-gather-vbnvb"]
Jul 21 13:27:42.210186 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:27:42.210162    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/2445b7a8-543f-4c05-8930-77167b7146f4-must-gather-output\") pod \"must-gather-vbnvb\" (UID: \"2445b7a8-543f-4c05-8930-77167b7146f4\") " pod="openshift-must-gather-s9mdb/must-gather-vbnvb"
Jul 21 13:27:42.210294 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:27:42.210209    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wg49g\" (UniqueName: \"kubernetes.io/projected/2445b7a8-543f-4c05-8930-77167b7146f4-kube-api-access-wg49g\") pod \"must-gather-vbnvb\" (UID: \"2445b7a8-543f-4c05-8930-77167b7146f4\") " pod="openshift-must-gather-s9mdb/must-gather-vbnvb"
Jul 21 13:27:42.311555 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:27:42.311533    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wg49g\" (UniqueName: \"kubernetes.io/projected/2445b7a8-543f-4c05-8930-77167b7146f4-kube-api-access-wg49g\") pod \"must-gather-vbnvb\" (UID: \"2445b7a8-543f-4c05-8930-77167b7146f4\") " pod="openshift-must-gather-s9mdb/must-gather-vbnvb"
Jul 21 13:27:42.311653 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:27:42.311622    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/2445b7a8-543f-4c05-8930-77167b7146f4-must-gather-output\") pod \"must-gather-vbnvb\" (UID: \"2445b7a8-543f-4c05-8930-77167b7146f4\") " pod="openshift-must-gather-s9mdb/must-gather-vbnvb"
Jul 21 13:27:42.311950 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:27:42.311929    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/2445b7a8-543f-4c05-8930-77167b7146f4-must-gather-output\") pod \"must-gather-vbnvb\" (UID: \"2445b7a8-543f-4c05-8930-77167b7146f4\") " pod="openshift-must-gather-s9mdb/must-gather-vbnvb"
Jul 21 13:27:42.327636 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:27:42.327616    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wg49g\" (UniqueName: \"kubernetes.io/projected/2445b7a8-543f-4c05-8930-77167b7146f4-kube-api-access-wg49g\") pod \"must-gather-vbnvb\" (UID: \"2445b7a8-543f-4c05-8930-77167b7146f4\") " pod="openshift-must-gather-s9mdb/must-gather-vbnvb"
Jul 21 13:27:42.404534 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:27:42.404482    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-s9mdb/must-gather-vbnvb"
Jul 21 13:27:42.546537 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:27:42.546513    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-s9mdb/must-gather-vbnvb"]
Jul 21 13:27:42.548579 ip-10-0-130-2 kubenswrapper[2556]: W0721 13:27:42.548553    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2445b7a8_543f_4c05_8930_77167b7146f4.slice/crio-eca476d6862e8ce5a7c4566fa2eab18f2130eac60a9171276ad3b1680b0125c8 WatchSource:0}: Error finding container eca476d6862e8ce5a7c4566fa2eab18f2130eac60a9171276ad3b1680b0125c8: Status 404 returned error can't find the container with id eca476d6862e8ce5a7c4566fa2eab18f2130eac60a9171276ad3b1680b0125c8
Jul 21 13:27:43.521250 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:27:43.521155    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-s9mdb/must-gather-vbnvb" event={"ID":"2445b7a8-543f-4c05-8930-77167b7146f4","Type":"ContainerStarted","Data":"e0037902604272410512fbe8fa828e9638642d01c63af2b9fd424a1337c40c40"}
Jul 21 13:27:43.521250 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:27:43.521208    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-s9mdb/must-gather-vbnvb" event={"ID":"2445b7a8-543f-4c05-8930-77167b7146f4","Type":"ContainerStarted","Data":"58151a3f84721a734b9574bbf5e7c7865e7024f179cceea2fabda5eb451454ea"}
Jul 21 13:27:43.521250 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:27:43.521226    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-s9mdb/must-gather-vbnvb" event={"ID":"2445b7a8-543f-4c05-8930-77167b7146f4","Type":"ContainerStarted","Data":"eca476d6862e8ce5a7c4566fa2eab18f2130eac60a9171276ad3b1680b0125c8"}
Jul 21 13:27:43.544779 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:27:43.544714    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-s9mdb/must-gather-vbnvb" podStartSLOduration=0.828165099 podStartE2EDuration="1.544692257s" totalImagesPullingTime="716.527158ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 13:27:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:27:43.541733006 +0000 UTC m=+2876.845738109" watchObservedRunningTime="2026-07-21 13:27:43.544692257 +0000 UTC m=+2876.848697348"
Jul 21 13:27:45.063287 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:27:45.063232    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-mchwp_d71014d4-ae0b-437e-adca-886a8ff5ac7f/global-pull-secret-syncer/0.log"
Jul 21 13:27:45.284084 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:27:45.284042    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-tpxr6_0f25175e-7fc8-497f-88f1-42b3d583bce2/konnectivity-agent/0.log"
Jul 21 13:27:45.430134 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:27:45.430043    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-130-2.ec2.internal_88a4b370840ce6eb63d3d3c786c424a4/haproxy/0.log"
Jul 21 13:27:50.542304 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:27:50.542275    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-pmr2h_7962da2c-145f-499e-82d3-be635b223b41/manager/0.log"
Jul 21 13:27:50.681143 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:27:50.681102    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-k5kss_b134efa5-61c2-46f9-bc3e-50cc5dfc5b61/registry-server/0.log"
Jul 21 13:27:51.147823 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:27:51.147782    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-xj2dd_ad723ec0-e63c-4a5e-acda-719533071ba2/manager/0.log"
Jul 21 13:27:52.787732 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:27:52.787671    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_2daab208-c37f-4c4e-bebd-516f511f318d/alertmanager/0.log"
Jul 21 13:27:52.822846 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:27:52.822790    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_2daab208-c37f-4c4e-bebd-516f511f318d/config-reloader/0.log"
Jul 21 13:27:52.858326 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:27:52.858294    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_2daab208-c37f-4c4e-bebd-516f511f318d/kube-rbac-proxy-web/0.log"
Jul 21 13:27:52.909005 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:27:52.908975    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_2daab208-c37f-4c4e-bebd-516f511f318d/kube-rbac-proxy/0.log"
Jul 21 13:27:52.966743 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:27:52.966714    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_2daab208-c37f-4c4e-bebd-516f511f318d/kube-rbac-proxy-metric/0.log"
Jul 21 13:27:53.010124 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:27:53.010091    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_2daab208-c37f-4c4e-bebd-516f511f318d/prom-label-proxy/0.log"
Jul 21 13:27:53.051820 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:27:53.051743    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_2daab208-c37f-4c4e-bebd-516f511f318d/init-config-reloader/0.log"
Jul 21 13:27:53.110803 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:27:53.110777    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-76cf8966f8-cldzr_a087fecd-1bf9-4449-a9a1-dc1aba2a2ec3/cluster-monitoring-operator/0.log"
Jul 21 13:27:53.175242 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:27:53.175214    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-j744v_eaaabf68-0d82-4671-a030-fead13f69633/kube-state-metrics/0.log"
Jul 21 13:27:53.206796 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:27:53.206759    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-j744v_eaaabf68-0d82-4671-a030-fead13f69633/kube-rbac-proxy-main/0.log"
Jul 21 13:27:53.239772 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:27:53.239740    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-j744v_eaaabf68-0d82-4671-a030-fead13f69633/kube-rbac-proxy-self/0.log"
Jul 21 13:27:53.314122 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:27:53.313988    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-7b7b7bc8bf-wx22h_b8bd4d42-5ed6-40eb-b934-83d26a1ce0ca/monitoring-plugin/0.log"
Jul 21 13:27:53.575893 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:27:53.575823    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-j6xgk_537b3296-9a40-42c6-9e84-456cc7f20f28/node-exporter/0.log"
Jul 21 13:27:53.607610 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:27:53.607582    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-j6xgk_537b3296-9a40-42c6-9e84-456cc7f20f28/kube-rbac-proxy/0.log"
Jul 21 13:27:53.641054 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:27:53.641031    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-j6xgk_537b3296-9a40-42c6-9e84-456cc7f20f28/init-textfile/0.log"
Jul 21 13:27:54.179626 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:27:54.179597    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-867bd765cb-6957j_ef6b7630-4103-4b25-82e2-249ce1c9f05f/telemeter-client/0.log"
Jul 21 13:27:54.247306 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:27:54.247283    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-867bd765cb-6957j_ef6b7630-4103-4b25-82e2-249ce1c9f05f/reload/0.log"
Jul 21 13:27:54.299369 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:27:54.299338    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-867bd765cb-6957j_ef6b7630-4103-4b25-82e2-249ce1c9f05f/kube-rbac-proxy/0.log"
Jul 21 13:27:54.775908 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:27:54.775871    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-s9mdb/perf-node-gather-daemonset-zx9kn"]
Jul 21 13:27:54.783997 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:27:54.783970    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-s9mdb/perf-node-gather-daemonset-zx9kn"
Jul 21 13:27:54.797039 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:27:54.793755    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-s9mdb/perf-node-gather-daemonset-zx9kn"]
Jul 21 13:27:54.963189 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:27:54.963151    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e2a1397f-c4fe-45bb-890e-1822a2a78e4b-sys\") pod \"perf-node-gather-daemonset-zx9kn\" (UID: \"e2a1397f-c4fe-45bb-890e-1822a2a78e4b\") " pod="openshift-must-gather-s9mdb/perf-node-gather-daemonset-zx9kn"
Jul 21 13:27:54.963381 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:27:54.963194    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mdrfg\" (UniqueName: \"kubernetes.io/projected/e2a1397f-c4fe-45bb-890e-1822a2a78e4b-kube-api-access-mdrfg\") pod \"perf-node-gather-daemonset-zx9kn\" (UID: \"e2a1397f-c4fe-45bb-890e-1822a2a78e4b\") " pod="openshift-must-gather-s9mdb/perf-node-gather-daemonset-zx9kn"
Jul 21 13:27:54.963381 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:27:54.963224    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e2a1397f-c4fe-45bb-890e-1822a2a78e4b-lib-modules\") pod \"perf-node-gather-daemonset-zx9kn\" (UID: \"e2a1397f-c4fe-45bb-890e-1822a2a78e4b\") " pod="openshift-must-gather-s9mdb/perf-node-gather-daemonset-zx9kn"
Jul 21 13:27:54.963381 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:27:54.963280    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e2a1397f-c4fe-45bb-890e-1822a2a78e4b-proc\") pod \"perf-node-gather-daemonset-zx9kn\" (UID: \"e2a1397f-c4fe-45bb-890e-1822a2a78e4b\") " pod="openshift-must-gather-s9mdb/perf-node-gather-daemonset-zx9kn"
Jul 21 13:27:54.963576 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:27:54.963373    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e2a1397f-c4fe-45bb-890e-1822a2a78e4b-podres\") pod \"perf-node-gather-daemonset-zx9kn\" (UID: \"e2a1397f-c4fe-45bb-890e-1822a2a78e4b\") " pod="openshift-must-gather-s9mdb/perf-node-gather-daemonset-zx9kn"
Jul 21 13:27:55.064606 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:27:55.064521    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e2a1397f-c4fe-45bb-890e-1822a2a78e4b-sys\") pod \"perf-node-gather-daemonset-zx9kn\" (UID: \"e2a1397f-c4fe-45bb-890e-1822a2a78e4b\") " pod="openshift-must-gather-s9mdb/perf-node-gather-daemonset-zx9kn"
Jul 21 13:27:55.064606 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:27:55.064565    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mdrfg\" (UniqueName: \"kubernetes.io/projected/e2a1397f-c4fe-45bb-890e-1822a2a78e4b-kube-api-access-mdrfg\") pod \"perf-node-gather-daemonset-zx9kn\" (UID: \"e2a1397f-c4fe-45bb-890e-1822a2a78e4b\") " pod="openshift-must-gather-s9mdb/perf-node-gather-daemonset-zx9kn"
Jul 21 13:27:55.064606 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:27:55.064585    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e2a1397f-c4fe-45bb-890e-1822a2a78e4b-lib-modules\") pod \"perf-node-gather-daemonset-zx9kn\" (UID: \"e2a1397f-c4fe-45bb-890e-1822a2a78e4b\") " pod="openshift-must-gather-s9mdb/perf-node-gather-daemonset-zx9kn"
Jul 21 13:27:55.064866 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:27:55.064642    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e2a1397f-c4fe-45bb-890e-1822a2a78e4b-proc\") pod \"perf-node-gather-daemonset-zx9kn\" (UID: \"e2a1397f-c4fe-45bb-890e-1822a2a78e4b\") " pod="openshift-must-gather-s9mdb/perf-node-gather-daemonset-zx9kn"
Jul 21 13:27:55.064866 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:27:55.064661    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e2a1397f-c4fe-45bb-890e-1822a2a78e4b-sys\") pod \"perf-node-gather-daemonset-zx9kn\" (UID: \"e2a1397f-c4fe-45bb-890e-1822a2a78e4b\") " pod="openshift-must-gather-s9mdb/perf-node-gather-daemonset-zx9kn"
Jul 21 13:27:55.064866 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:27:55.064673    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e2a1397f-c4fe-45bb-890e-1822a2a78e4b-podres\") pod \"perf-node-gather-daemonset-zx9kn\" (UID: \"e2a1397f-c4fe-45bb-890e-1822a2a78e4b\") " pod="openshift-must-gather-s9mdb/perf-node-gather-daemonset-zx9kn"
Jul 21 13:27:55.064866 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:27:55.064785    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e2a1397f-c4fe-45bb-890e-1822a2a78e4b-lib-modules\") pod \"perf-node-gather-daemonset-zx9kn\" (UID: \"e2a1397f-c4fe-45bb-890e-1822a2a78e4b\") " pod="openshift-must-gather-s9mdb/perf-node-gather-daemonset-zx9kn"
Jul 21 13:27:55.064866 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:27:55.064789    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e2a1397f-c4fe-45bb-890e-1822a2a78e4b-podres\") pod \"perf-node-gather-daemonset-zx9kn\" (UID: \"e2a1397f-c4fe-45bb-890e-1822a2a78e4b\") " pod="openshift-must-gather-s9mdb/perf-node-gather-daemonset-zx9kn"
Jul 21 13:27:55.064866 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:27:55.064838    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e2a1397f-c4fe-45bb-890e-1822a2a78e4b-proc\") pod \"perf-node-gather-daemonset-zx9kn\" (UID: \"e2a1397f-c4fe-45bb-890e-1822a2a78e4b\") " pod="openshift-must-gather-s9mdb/perf-node-gather-daemonset-zx9kn"
Jul 21 13:27:55.075671 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:27:55.075643    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mdrfg\" (UniqueName: \"kubernetes.io/projected/e2a1397f-c4fe-45bb-890e-1822a2a78e4b-kube-api-access-mdrfg\") pod \"perf-node-gather-daemonset-zx9kn\" (UID: \"e2a1397f-c4fe-45bb-890e-1822a2a78e4b\") " pod="openshift-must-gather-s9mdb/perf-node-gather-daemonset-zx9kn"
Jul 21 13:27:55.107558 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:27:55.107520    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-s9mdb/perf-node-gather-daemonset-zx9kn"
Jul 21 13:27:55.301481 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:27:55.299831    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-s9mdb/perf-node-gather-daemonset-zx9kn"]
Jul 21 13:27:55.301869 ip-10-0-130-2 kubenswrapper[2556]: W0721 13:27:55.301565    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pode2a1397f_c4fe_45bb_890e_1822a2a78e4b.slice/crio-1d277791537dd2fa9a50b8f2b365e2207e79d03b4782b387abfb3b7c11c3e4f6 WatchSource:0}: Error finding container 1d277791537dd2fa9a50b8f2b365e2207e79d03b4782b387abfb3b7c11c3e4f6: Status 404 returned error can't find the container with id 1d277791537dd2fa9a50b8f2b365e2207e79d03b4782b387abfb3b7c11c3e4f6
Jul 21 13:27:55.633389 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:27:55.633297    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-s9mdb/perf-node-gather-daemonset-zx9kn" event={"ID":"e2a1397f-c4fe-45bb-890e-1822a2a78e4b","Type":"ContainerStarted","Data":"d4f67a9e28a43f2206341466481647e3a133494b9474b022636beb651b7cb719"}
Jul 21 13:27:55.633389 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:27:55.633369    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-s9mdb/perf-node-gather-daemonset-zx9kn" event={"ID":"e2a1397f-c4fe-45bb-890e-1822a2a78e4b","Type":"ContainerStarted","Data":"1d277791537dd2fa9a50b8f2b365e2207e79d03b4782b387abfb3b7c11c3e4f6"}
Jul 21 13:27:55.633615 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:27:55.633595    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-s9mdb/perf-node-gather-daemonset-zx9kn"
Jul 21 13:27:55.661690 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:27:55.661641    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-s9mdb/perf-node-gather-daemonset-zx9kn" podStartSLOduration=1.6616259740000001 podStartE2EDuration="1.661625974s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 13:27:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:27:55.661399713 +0000 UTC m=+2888.965404803" watchObservedRunningTime="2026-07-21 13:27:55.661625974 +0000 UTC m=+2888.965631065"
Jul 21 13:27:56.969207 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:27:56.969176    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-fbbbc87c4-mgm6t_d3b4d45b-90db-4272-9d7b-995c91fa468e/console/0.log"
Jul 21 13:27:57.760057 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:27:57.760029    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-78cf4c6f6-wd79x_95665b69-15a8-4324-adee-a9bf0a9f0865/volume-data-source-validator/0.log"
Jul 21 13:27:59.059330 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:27:59.059299    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-94clr_eb9d39d0-f6a2-4050-a227-8aa6061b4799/dns/0.log"
Jul 21 13:27:59.099294 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:27:59.099263    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-94clr_eb9d39d0-f6a2-4050-a227-8aa6061b4799/kube-rbac-proxy/0.log"
Jul 21 13:27:59.340697 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:27:59.340623    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-z24v8_e5faf5b4-5f7e-44dd-9503-3cc15f07533e/dns-node-resolver/0.log"
Jul 21 13:28:00.004292 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:28:00.004264    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-f4c5cdd74-l4dbz_755f9627-4c80-4597-ac00-670d261bb1ba/registry/0.log"
Jul 21 13:28:00.047756 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:28:00.047733    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-76wr9_5adf7abd-fea9-4a2a-8f96-a73bd0a3404a/node-ca/0.log"
Jul 21 13:28:01.134483 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:28:01.134453    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4gldrd_856c41ff-2032-4fbd-91a2-44a28c8682ce/istio-proxy/0.log"
Jul 21 13:28:01.320043 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:28:01.319992    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-67cb6d6b5d-rfwlt_041ed91e-0571-4917-a641-c554b8734229/kube-auth-proxy/0.log"
Jul 21 13:28:01.354064 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:28:01.354032    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-b55kr_fe4ed9c9-5046-4bb2-acbd-8d83d8b8eb2c/istio-proxy/0.log"
Jul 21 13:28:01.390472 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:28:01.390391    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-b55kr_fe4ed9c9-5046-4bb2-acbd-8d83d8b8eb2c/istio-proxy/1.log"
Jul 21 13:28:01.427846 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:28:01.427820    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-xmqdx_ed44480e-4c2e-4539-8f87-d6af31005e3d/payload-pre-processing/0.log"
Jul 21 13:28:01.463066 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:28:01.463036    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-q5zgh_0e9d6e83-1286-4634-866a-3ffcdcb84dbc/payload-processing/0.log"
Jul 21 13:28:01.651594 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:28:01.651510    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-s9mdb/perf-node-gather-daemonset-zx9kn"
Jul 21 13:28:02.280649 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:28:02.280625    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-8kzkx_9025d3cc-72d5-4cdb-aad8-79186d9a48ec/serve-healthcheck-canary/0.log"
Jul 21 13:28:02.873736 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:28:02.873706    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-555d9d7885-rn8m5_277500e3-0956-4360-90a2-6512c5725349/insights-operator/0.log"
Jul 21 13:28:02.874650 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:28:02.874618    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-555d9d7885-rn8m5_277500e3-0956-4360-90a2-6512c5725349/insights-operator/1.log"
Jul 21 13:28:02.900281 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:28:02.900254    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-9j6f5_dfe654b2-569e-4c88-a935-0f82db6839e5/kube-rbac-proxy/0.log"
Jul 21 13:28:02.925982 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:28:02.925950    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-9j6f5_dfe654b2-569e-4c88-a935-0f82db6839e5/exporter/0.log"
Jul 21 13:28:02.952928 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:28:02.952898    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-9j6f5_dfe654b2-569e-4c88-a935-0f82db6839e5/extractor/0.log"
Jul 21 13:28:05.356393 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:28:05.356367    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4gldrd_856c41ff-2032-4fbd-91a2-44a28c8682ce/istio-proxy/0.log"
Jul 21 13:28:05.518134 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:28:05.518110    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-67cb6d6b5d-rfwlt_041ed91e-0571-4917-a641-c554b8734229/kube-auth-proxy/0.log"
Jul 21 13:28:05.547810 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:28:05.547776    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-b55kr_fe4ed9c9-5046-4bb2-acbd-8d83d8b8eb2c/istio-proxy/0.log"
Jul 21 13:28:05.581446 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:28:05.581414    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-b55kr_fe4ed9c9-5046-4bb2-acbd-8d83d8b8eb2c/istio-proxy/1.log"
Jul 21 13:28:05.609390 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:28:05.609308    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-xmqdx_ed44480e-4c2e-4539-8f87-d6af31005e3d/payload-pre-processing/0.log"
Jul 21 13:28:05.627500 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:28:05.627480    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-6r5kh_68425978-0369-4748-ba6f-b3cac04a2552/manager/0.log"
Jul 21 13:28:05.635411 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:28:05.635366    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-q5zgh_0e9d6e83-1286-4634-866a-3ffcdcb84dbc/payload-processing/0.log"
Jul 21 13:28:05.891365 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:28:05.891285    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-94vcn_7c34ea1b-966c-490f-9cbb-f9ff00bc95d8/manager/0.log"
Jul 21 13:28:06.036220 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:28:06.036186    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-phbht_a9b7bd6c-d5fa-4c79-890e-7509ba41dd4f/server/0.log"
Jul 21 13:28:06.133856 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:28:06.133823    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-s7tkd_7bc3e182-acec-4866-9dd0-f3db1e521186/manager/0.log"
Jul 21 13:28:06.353893 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:28:06.353832    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-579d89bddd-trnmc_226755f7-2596-454a-a055-36be3e543967/manager/0.log"
Jul 21 13:28:06.379907 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:28:06.379884    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-579d89bddd-trnmc_226755f7-2596-454a-a055-36be3e543967/copy-manifests/0.log"
Jul 21 13:28:06.959860 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:28:06.959834    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4gldrd_856c41ff-2032-4fbd-91a2-44a28c8682ce/istio-proxy/0.log"
Jul 21 13:28:07.119237 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:28:07.119168    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-67cb6d6b5d-rfwlt_041ed91e-0571-4917-a641-c554b8734229/kube-auth-proxy/0.log"
Jul 21 13:28:07.141852 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:28:07.141827    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-b55kr_fe4ed9c9-5046-4bb2-acbd-8d83d8b8eb2c/istio-proxy/0.log"
Jul 21 13:28:07.176164 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:28:07.176133    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-b55kr_fe4ed9c9-5046-4bb2-acbd-8d83d8b8eb2c/istio-proxy/1.log"
Jul 21 13:28:07.203734 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:28:07.203703    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-xmqdx_ed44480e-4c2e-4539-8f87-d6af31005e3d/payload-pre-processing/0.log"
Jul 21 13:28:07.232367 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:28:07.232340    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-q5zgh_0e9d6e83-1286-4634-866a-3ffcdcb84dbc/payload-processing/0.log"
Jul 21 13:28:07.642545 ip-10-0-130-2 kubenswrapper[2556]: I0721 13:28:07.642520    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-xhrm6_b3312e1f-054d-4e08-863b-bf3c5dd54a73/openshift-lws-operator/0.log"