Jul 16 11:03:27.239945 ip-10-0-129-28 systemd[1]: Starting Kubernetes Kubelet...
Jul 16 11:03:28.987189 ip-10-0-129-28 kubenswrapper[2583]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 16 11:03:28.987189 ip-10-0-129-28 kubenswrapper[2583]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 16 11:03:28.987189 ip-10-0-129-28 kubenswrapper[2583]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 16 11:03:28.987189 ip-10-0-129-28 kubenswrapper[2583]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 16 11:03:28.987189 ip-10-0-129-28 kubenswrapper[2583]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 16 11:03:28.989070 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:28.988966    2583 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 16 11:03:28.992991 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.992969    2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 16 11:03:28.992991 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.992989    2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 16 11:03:28.992991 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.992993    2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 16 11:03:28.993101 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.992997    2583 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 16 11:03:28.993101 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.993001    2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 16 11:03:28.993101 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.993005    2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 16 11:03:28.993101 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.993008    2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 16 11:03:28.993101 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.993011    2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 16 11:03:28.993101 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.993014    2583 feature_gate.go:328] unrecognized feature gate: Example2
Jul 16 11:03:28.993101 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.993017    2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 16 11:03:28.993101 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.993020    2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 16 11:03:28.993101 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.993030    2583 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 16 11:03:28.993101 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.993033    2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 16 11:03:28.993101 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.993036    2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 16 11:03:28.993101 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.993039    2583 feature_gate.go:328] unrecognized feature gate: Example
Jul 16 11:03:28.993101 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.993041    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 16 11:03:28.993101 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.993044    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 16 11:03:28.993101 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.993047    2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 16 11:03:28.993101 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.993050    2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 16 11:03:28.993101 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.993052    2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 16 11:03:28.993101 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.993055    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 16 11:03:28.993101 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.993058    2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 16 11:03:28.993101 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.993061    2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 16 11:03:28.993584 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.993063    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 16 11:03:28.993584 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.993066    2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 16 11:03:28.993584 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.993069    2583 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 16 11:03:28.993584 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.993072    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 16 11:03:28.993584 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.993075    2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 16 11:03:28.993584 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.993078    2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 16 11:03:28.993584 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.993081    2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 16 11:03:28.993584 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.993083    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 16 11:03:28.993584 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.993086    2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 16 11:03:28.993584 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.993089    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 16 11:03:28.993584 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.993091    2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 16 11:03:28.993584 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.993095    2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 16 11:03:28.993584 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.993097    2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 16 11:03:28.993584 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.993101    2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 16 11:03:28.993584 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.993104    2583 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 16 11:03:28.993584 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.993107    2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 16 11:03:28.993584 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.993111    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 16 11:03:28.993584 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.993114    2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 16 11:03:28.993584 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.993117    2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 16 11:03:28.993584 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.993120    2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 16 11:03:28.994108 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.993123    2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 16 11:03:28.994108 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.993126    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 16 11:03:28.994108 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.993128    2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 16 11:03:28.994108 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.993131    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 16 11:03:28.994108 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.993134    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 16 11:03:28.994108 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.993136    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 16 11:03:28.994108 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.993139    2583 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 16 11:03:28.994108 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.993143    2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 16 11:03:28.994108 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.993145    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 16 11:03:28.994108 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.993148    2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 16 11:03:28.994108 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.993150    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 16 11:03:28.994108 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.993155    2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 16 11:03:28.994108 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.993160    2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 16 11:03:28.994108 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.993164    2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 16 11:03:28.994108 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.993167    2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 16 11:03:28.994108 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.993170    2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 16 11:03:28.994108 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.993173    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 16 11:03:28.994108 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.993176    2583 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 16 11:03:28.994108 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.993179    2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 16 11:03:28.994583 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.993182    2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 16 11:03:28.994583 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.993184    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 16 11:03:28.994583 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.993189    2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 16 11:03:28.994583 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.993192    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 16 11:03:28.994583 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.993195    2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 16 11:03:28.994583 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.993199    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 16 11:03:28.994583 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.993203    2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 16 11:03:28.994583 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.993207    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 16 11:03:28.994583 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.993210    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 16 11:03:28.994583 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.993213    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 16 11:03:28.994583 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.993216    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 16 11:03:28.994583 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.993219    2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 16 11:03:28.994583 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.993222    2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 16 11:03:28.994583 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.993225    2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 16 11:03:28.994583 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.993228    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 16 11:03:28.994583 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.993232    2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 16 11:03:28.994583 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.993236    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 16 11:03:28.994583 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.993239    2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 16 11:03:28.994583 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.993243    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 16 11:03:28.995088 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.993246    2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 16 11:03:28.995088 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.993250    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 16 11:03:28.995088 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.993252    2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 16 11:03:28.995088 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.993255    2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 16 11:03:28.995088 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.993257    2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 16 11:03:28.995088 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.993260    2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 16 11:03:28.995088 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.993263    2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 16 11:03:28.995088 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.993266    2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 16 11:03:28.995088 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.993268    2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 16 11:03:28.995088 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.993271    2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 16 11:03:28.995088 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.993274    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 16 11:03:28.995088 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.993277    2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 16 11:03:28.995088 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.993279    2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 16 11:03:28.995088 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.993282    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 16 11:03:28.995088 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.993285    2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 16 11:03:28.995088 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.993288    2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 16 11:03:28.995088 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.993291    2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 16 11:03:28.995088 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.993294    2583 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 16 11:03:28.995088 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.994108    2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 16 11:03:28.995088 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.994116    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 16 11:03:28.995686 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.994120    2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 16 11:03:28.995686 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.994123    2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 16 11:03:28.995686 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.994126    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 16 11:03:28.995686 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.994130    2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 16 11:03:28.995686 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.994134    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 16 11:03:28.995686 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.994136    2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 16 11:03:28.995686 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.994139    2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 16 11:03:28.995686 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.994142    2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 16 11:03:28.995686 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.994145    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 16 11:03:28.995686 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.994148    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 16 11:03:28.995686 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.994151    2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 16 11:03:28.995686 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.994153    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 16 11:03:28.995686 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.994156    2583 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 16 11:03:28.995686 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.994159    2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 16 11:03:28.995686 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.994162    2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 16 11:03:28.995686 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.994164    2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 16 11:03:28.995686 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.994167    2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 16 11:03:28.995686 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.994170    2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 16 11:03:28.995686 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.994173    2583 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 16 11:03:28.996197 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.994176    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 16 11:03:28.996197 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.994179    2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 16 11:03:28.996197 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.994182    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 16 11:03:28.996197 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.994185    2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 16 11:03:28.996197 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.994188    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 16 11:03:28.996197 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.994192    2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 16 11:03:28.996197 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.994196    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 16 11:03:28.996197 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.994198    2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 16 11:03:28.996197 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.994201    2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 16 11:03:28.996197 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.994204    2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 16 11:03:28.996197 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.994207    2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 16 11:03:28.996197 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.994210    2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 16 11:03:28.996197 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.994212    2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 16 11:03:28.996197 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.994215    2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 16 11:03:28.996197 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.994218    2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 16 11:03:28.996197 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.994221    2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 16 11:03:28.996197 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.994224    2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 16 11:03:28.996197 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.994227    2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 16 11:03:28.996197 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.994230    2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 16 11:03:28.996690 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.994233    2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 16 11:03:28.996690 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.994235    2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 16 11:03:28.996690 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.994239    2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 16 11:03:28.996690 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.994241    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 16 11:03:28.996690 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.994244    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 16 11:03:28.996690 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.994248    2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 16 11:03:28.996690 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.994250    2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 16 11:03:28.996690 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.994253    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 16 11:03:28.996690 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.994258    2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 16 11:03:28.996690 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.994262    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 16 11:03:28.996690 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.994266    2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 16 11:03:28.996690 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.994269    2583 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 16 11:03:28.996690 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.994272    2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 16 11:03:28.996690 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.994276    2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 16 11:03:28.996690 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.994279    2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 16 11:03:28.996690 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.994282    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 16 11:03:28.996690 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.994285    2583 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 16 11:03:28.996690 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.994288    2583 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 16 11:03:28.996690 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.994291    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 16 11:03:28.996690 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.994294    2583 feature_gate.go:328] unrecognized feature gate: Example
Jul 16 11:03:28.997209 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.994297    2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 16 11:03:28.997209 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.994299    2583 feature_gate.go:328] unrecognized feature gate: Example2
Jul 16 11:03:28.997209 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.994303    2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 16 11:03:28.997209 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.994305    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 16 11:03:28.997209 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.994308    2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 16 11:03:28.997209 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.994311    2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 16 11:03:28.997209 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.994314    2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 16 11:03:28.997209 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.994317    2583 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 16 11:03:28.997209 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.994319    2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 16 11:03:28.997209 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.994322    2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 16 11:03:28.997209 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.994325    2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 16 11:03:28.997209 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.994330    2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 16 11:03:28.997209 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.994334    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 16 11:03:28.997209 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.994337    2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 16 11:03:28.997209 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.994341    2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 16 11:03:28.997209 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.994344    2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 16 11:03:28.997209 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.994347    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 16 11:03:28.997209 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.994350    2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 16 11:03:28.997209 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.994353    2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 16 11:03:28.997209 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.994356    2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 16 11:03:28.997756 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.994359    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 16 11:03:28.997756 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.994362    2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 16 11:03:28.997756 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.994366    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 16 11:03:28.997756 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.994369    2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 16 11:03:28.997756 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.994372    2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 16 11:03:28.997756 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.994374    2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 16 11:03:28.997756 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.994377    2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 16 11:03:28.997756 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.994380    2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 16 11:03:28.997756 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.994383    2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 16 11:03:28.997756 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.994386    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 16 11:03:28.997756 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.994389    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 16 11:03:28.997756 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.994391    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 16 11:03:28.997756 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.994394    2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 16 11:03:28.997756 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.994397    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 16 11:03:28.997756 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.994399    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 16 11:03:28.997756 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.994402    2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 16 11:03:28.997756 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.994405    2583 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 16 11:03:28.997756 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.994407    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 16 11:03:28.997756 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.994410    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 16 11:03:28.997756 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.994855    2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 16 11:03:28.998258 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.994861    2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 16 11:03:28.998258 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.994864    2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 16 11:03:28.998258 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.994867    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 16 11:03:28.998258 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.994870    2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 16 11:03:28.998258 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.994873    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 16 11:03:28.998258 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.994877    2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 16 11:03:28.998258 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.994880    2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 16 11:03:28.998258 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.994884    2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 16 11:03:28.998258 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.994887    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 16 11:03:28.998258 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.994889    2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 16 11:03:28.998258 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.994892    2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 16 11:03:28.998258 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.994895    2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 16 11:03:28.998258 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.994898    2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 16 11:03:28.998258 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.994901    2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 16 11:03:28.998258 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.994909    2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 16 11:03:28.998258 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.994911    2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 16 11:03:28.998258 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.994914    2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 16 11:03:28.998258 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.994917    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 16 11:03:28.998258 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.994920    2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 16 11:03:28.998258 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.994923    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 16 11:03:28.998799 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.994926    2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 16 11:03:28.998799 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.994930    2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 16 11:03:28.998799 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.994933    2583 feature_gate.go:328] unrecognized feature gate: Example2
Jul 16 11:03:28.998799 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.994936    2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 16 11:03:28.998799 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.994939    2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 16 11:03:28.998799 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.994942    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 16 11:03:28.998799 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.994944    2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 16 11:03:28.998799 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.994947    2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 16 11:03:28.998799 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.994950    2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 16 11:03:28.998799 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.994953    2583 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 16 11:03:28.998799 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.994955    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 16 11:03:28.998799 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.994960    2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 16 11:03:28.998799 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.994964    2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 16 11:03:28.998799 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.994967    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 16 11:03:28.998799 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.994970    2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 16 11:03:28.998799 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.994973    2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 16 11:03:28.998799 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.994977    2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 16 11:03:28.998799 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.994980    2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 16 11:03:28.998799 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.994983    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 16 11:03:28.998799 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.994986    2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 16 11:03:28.999301 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.994988    2583 feature_gate.go:328] unrecognized feature gate: Example
Jul 16 11:03:28.999301 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.994991    2583 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 16 11:03:28.999301 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.994994    2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 16 11:03:28.999301 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.994997    2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 16 11:03:28.999301 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.995001    2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 16 11:03:28.999301 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.995003    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 16 11:03:28.999301 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.995007    2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 16 11:03:28.999301 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.995010    2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 16 11:03:28.999301 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.995013    2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 16 11:03:28.999301 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.995016    2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 16 11:03:28.999301 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.995019    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 16 11:03:28.999301 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.995022    2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 16 11:03:28.999301 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.995024    2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 16 11:03:28.999301 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.995027    2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 16 11:03:28.999301 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.995030    2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 16 11:03:28.999301 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.995033    2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 16 11:03:28.999301 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.995036    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 16 11:03:28.999301 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.995039    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 16 11:03:28.999301 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.995042    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 16 11:03:28.999301 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.995045    2583 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 16 11:03:28.999814 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.995047    2583 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 16 11:03:28.999814 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.995050    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 16 11:03:28.999814 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.995053    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 16 11:03:28.999814 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.995055    2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 16 11:03:28.999814 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.995058    2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 16 11:03:28.999814 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.995061    2583 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 16 11:03:28.999814 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.995064    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 16 11:03:28.999814 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.995066    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 16 11:03:28.999814 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.995069    2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 16 11:03:28.999814 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.995072    2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 16 11:03:28.999814 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.995075    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 16 11:03:28.999814 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.995078    2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 16 11:03:28.999814 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.995080    2583 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 16 11:03:28.999814 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.995083    2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 16 11:03:28.999814 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.995086    2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 16 11:03:28.999814 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.995089    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 16 11:03:28.999814 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.995092    2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 16 11:03:28.999814 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.995095    2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 16 11:03:28.999814 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.995100    2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 16 11:03:28.999814 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.995103    2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 16 11:03:29.000366 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.995106    2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 16 11:03:29.000366 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.995109    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 16 11:03:29.000366 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.995111    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 16 11:03:29.000366 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.995114    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 16 11:03:29.000366 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.995118    2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 16 11:03:29.000366 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.995121    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 16 11:03:29.000366 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.995124    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 16 11:03:29.000366 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.995127    2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 16 11:03:29.000366 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.995130    2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 16 11:03:29.000366 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.995132    2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 16 11:03:29.000366 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.995135    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 16 11:03:29.000366 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.995138    2583 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 16 11:03:29.000366 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.995141    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 16 11:03:29.000366 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.995143    2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 16 11:03:29.000366 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.995147    2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 16 11:03:29.000366 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.995151    2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 16 11:03:29.000366 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.995154    2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 16 11:03:29.000366 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.995157    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 16 11:03:29.000366 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:28.995243    2583 flags.go:64] FLAG: --address="0.0.0.0"
Jul 16 11:03:29.000866 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:28.995251    2583 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 16 11:03:29.000866 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:28.995257    2583 flags.go:64] FLAG: --anonymous-auth="true"
Jul 16 11:03:29.000866 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:28.995264    2583 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 16 11:03:29.000866 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:28.995269    2583 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 16 11:03:29.000866 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:28.995272    2583 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 16 11:03:29.000866 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:28.995277    2583 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 16 11:03:29.000866 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:28.995281    2583 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 16 11:03:29.000866 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:28.995285    2583 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 16 11:03:29.000866 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:28.995288    2583 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 16 11:03:29.000866 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:28.995292    2583 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 16 11:03:29.000866 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:28.995296    2583 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 16 11:03:29.000866 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:28.995300    2583 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 16 11:03:29.000866 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:28.995303    2583 flags.go:64] FLAG: --cgroup-root=""
Jul 16 11:03:29.000866 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:28.995306    2583 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 16 11:03:29.000866 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:28.995309    2583 flags.go:64] FLAG: --client-ca-file=""
Jul 16 11:03:29.000866 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:28.995312    2583 flags.go:64] FLAG: --cloud-provider="external"
Jul 16 11:03:29.000866 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:28.995316    2583 flags.go:64] FLAG: --cluster-dns="[]"
Jul 16 11:03:29.000866 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:28.995321    2583 flags.go:64] FLAG: --cluster-domain=""
Jul 16 11:03:29.000866 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:28.995324    2583 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 16 11:03:29.000866 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:28.995327    2583 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 16 11:03:29.000866 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:28.995331    2583 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 16 11:03:29.000866 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:28.995334    2583 flags.go:64] FLAG: --container-log-max-files="5"
Jul 16 11:03:29.000866 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:28.995338    2583 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 16 11:03:29.000866 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:28.995341    2583 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 16 11:03:29.001457 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:28.995345    2583 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 16 11:03:29.001457 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:28.995348    2583 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 16 11:03:29.001457 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:28.995351    2583 flags.go:64] FLAG: --contention-profiling="false"
Jul 16 11:03:29.001457 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:28.995355    2583 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 16 11:03:29.001457 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:28.995358    2583 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 16 11:03:29.001457 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:28.995361    2583 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 16 11:03:29.001457 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:28.995364    2583 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 16 11:03:29.001457 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:28.995369    2583 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 16 11:03:29.001457 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:28.995372    2583 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 16 11:03:29.001457 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:28.995375    2583 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 16 11:03:29.001457 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:28.995380    2583 flags.go:64] FLAG: --enable-load-reader="false"
Jul 16 11:03:29.001457 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:28.995383    2583 flags.go:64] FLAG: --enable-server="true"
Jul 16 11:03:29.001457 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:28.995387    2583 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 16 11:03:29.001457 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:28.995392    2583 flags.go:64] FLAG: --event-burst="100"
Jul 16 11:03:29.001457 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:28.995395    2583 flags.go:64] FLAG: --event-qps="50"
Jul 16 11:03:29.001457 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:28.995398    2583 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 16 11:03:29.001457 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:28.995401    2583 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 16 11:03:29.001457 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:28.995404    2583 flags.go:64] FLAG: --eviction-hard=""
Jul 16 11:03:29.001457 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:28.995408    2583 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 16 11:03:29.001457 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:28.995412    2583 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 16 11:03:29.001457 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:28.995415    2583 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 16 11:03:29.001457 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:28.995419    2583 flags.go:64] FLAG: --eviction-soft=""
Jul 16 11:03:29.001457 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:28.995422    2583 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 16 11:03:29.001457 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:28.995425    2583 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 16 11:03:29.001457 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:28.995428    2583 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 16 11:03:29.002177 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:28.995431    2583 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 16 11:03:29.002177 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:28.995434    2583 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 16 11:03:29.002177 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:28.995437    2583 flags.go:64] FLAG: --fail-swap-on="true"
Jul 16 11:03:29.002177 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:28.995441    2583 flags.go:64] FLAG: --feature-gates=""
Jul 16 11:03:29.002177 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:28.995445    2583 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 16 11:03:29.002177 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:28.995448    2583 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 16 11:03:29.002177 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:28.995452    2583 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 16 11:03:29.002177 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:28.995456    2583 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 16 11:03:29.002177 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:28.995459    2583 flags.go:64] FLAG: --healthz-port="10248"
Jul 16 11:03:29.002177 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:28.995462    2583 flags.go:64] FLAG: --help="false"
Jul 16 11:03:29.002177 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:28.995465    2583 flags.go:64] FLAG: --hostname-override="ip-10-0-129-28.ec2.internal"
Jul 16 11:03:29.002177 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:28.995469    2583 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 16 11:03:29.002177 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:28.995472    2583 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 16 11:03:29.002177 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:28.995475    2583 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 16 11:03:29.002177 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:28.995479    2583 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 16 11:03:29.002177 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:28.995483    2583 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 16 11:03:29.002177 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:28.995486    2583 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 16 11:03:29.002177 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:28.995491    2583 flags.go:64] FLAG: --image-service-endpoint=""
Jul 16 11:03:29.002177 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:28.995494    2583 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 16 11:03:29.002177 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:28.995497    2583 flags.go:64] FLAG: --kube-api-burst="100"
Jul 16 11:03:29.002177 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:28.995500    2583 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 16 11:03:29.002177 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:28.995504    2583 flags.go:64] FLAG: --kube-api-qps="50"
Jul 16 11:03:29.002177 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:28.995507    2583 flags.go:64] FLAG: --kube-reserved=""
Jul 16 11:03:29.002177 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:28.995510    2583 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 16 11:03:29.003187 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:28.995513    2583 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 16 11:03:29.003187 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:28.995516    2583 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 16 11:03:29.003187 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:28.995520    2583 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 16 11:03:29.003187 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:28.995523    2583 flags.go:64] FLAG: --lock-file=""
Jul 16 11:03:29.003187 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:28.995526    2583 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 16 11:03:29.003187 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:28.995529    2583 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 16 11:03:29.003187 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:28.995532    2583 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 16 11:03:29.003187 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:28.995539    2583 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 16 11:03:29.003187 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:28.995542    2583 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 16 11:03:29.003187 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:28.995545    2583 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 16 11:03:29.003187 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:28.995548    2583 flags.go:64] FLAG: --logging-format="text"
Jul 16 11:03:29.003187 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:28.995551    2583 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 16 11:03:29.003187 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:28.995555    2583 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 16 11:03:29.003187 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:28.995558    2583 flags.go:64] FLAG: --manifest-url=""
Jul 16 11:03:29.003187 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:28.995560    2583 flags.go:64] FLAG: --manifest-url-header=""
Jul 16 11:03:29.003187 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:28.995566    2583 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 16 11:03:29.003187 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:28.995569    2583 flags.go:64] FLAG: --max-open-files="1000000"
Jul 16 11:03:29.003187 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:28.995573    2583 flags.go:64] FLAG: --max-pods="110"
Jul 16 11:03:29.003187 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:28.995576    2583 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 16 11:03:29.003187 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:28.995580    2583 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 16 11:03:29.003187 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:28.995583    2583 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 16 11:03:29.003187 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:28.995586    2583 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 16 11:03:29.003187 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:28.995589    2583 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 16 11:03:29.003187 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:28.995592    2583 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 16 11:03:29.003187 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:28.995595    2583 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 16 11:03:29.004329 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:28.995605    2583 flags.go:64] FLAG: --node-status-max-images="50"
Jul 16 11:03:29.004329 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:28.995608    2583 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 16 11:03:29.004329 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:28.995611    2583 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 16 11:03:29.004329 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:28.995615    2583 flags.go:64] FLAG: --pod-cidr=""
Jul 16 11:03:29.004329 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:28.995618    2583 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926"
Jul 16 11:03:29.004329 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:28.995625    2583 flags.go:64] FLAG: --pod-manifest-path=""
Jul 16 11:03:29.004329 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:28.995628    2583 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 16 11:03:29.004329 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:28.995631    2583 flags.go:64] FLAG: --pods-per-core="0"
Jul 16 11:03:29.004329 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:28.995634    2583 flags.go:64] FLAG: --port="10250"
Jul 16 11:03:29.004329 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:28.995637    2583 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 16 11:03:29.004329 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:28.995641    2583 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-06253ddacb365684e"
Jul 16 11:03:29.004329 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:28.995644    2583 flags.go:64] FLAG: --qos-reserved=""
Jul 16 11:03:29.004329 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:28.995647    2583 flags.go:64] FLAG: --read-only-port="10255"
Jul 16 11:03:29.004329 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:28.995650    2583 flags.go:64] FLAG: --register-node="true"
Jul 16 11:03:29.004329 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:28.995654    2583 flags.go:64] FLAG: --register-with-taints=""
Jul 16 11:03:29.004329 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:28.995657    2583 flags.go:64] FLAG: --registry-burst="10"
Jul 16 11:03:29.004329 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:28.995660    2583 flags.go:64] FLAG: --registry-qps="5"
Jul 16 11:03:29.004329 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:28.995664    2583 flags.go:64] FLAG: --reserved-cpus=""
Jul 16 11:03:29.004329 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:28.995667    2583 flags.go:64] FLAG: --reserved-memory=""
Jul 16 11:03:29.004329 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:28.995684    2583 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 16 11:03:29.004329 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:28.995688    2583 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 16 11:03:29.004329 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:28.995691    2583 flags.go:64] FLAG: --rotate-certificates="false"
Jul 16 11:03:29.004329 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:28.995694    2583 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 16 11:03:29.004329 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:28.995698    2583 flags.go:64] FLAG: --runonce="false"
Jul 16 11:03:29.004329 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:28.995701    2583 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 16 11:03:29.005464 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:28.995705    2583 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 16 11:03:29.005464 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:28.995708    2583 flags.go:64] FLAG: --seccomp-default="false"
Jul 16 11:03:29.005464 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:28.995711    2583 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 16 11:03:29.005464 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:28.995714    2583 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 16 11:03:29.005464 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:28.995718    2583 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 16 11:03:29.005464 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:28.995721    2583 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 16 11:03:29.005464 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:28.995725    2583 flags.go:64] FLAG: --storage-driver-password="root"
Jul 16 11:03:29.005464 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:28.995729    2583 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 16 11:03:29.005464 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:28.995732    2583 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 16 11:03:29.005464 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:28.995736    2583 flags.go:64] FLAG: --storage-driver-user="root"
Jul 16 11:03:29.005464 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:28.995738    2583 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 16 11:03:29.005464 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:28.995742    2583 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 16 11:03:29.005464 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:28.995746    2583 flags.go:64] FLAG: --system-cgroups=""
Jul 16 11:03:29.005464 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:28.995749    2583 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 16 11:03:29.005464 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:28.995754    2583 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 16 11:03:29.005464 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:28.995757    2583 flags.go:64] FLAG: --tls-cert-file=""
Jul 16 11:03:29.005464 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:28.995760    2583 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 16 11:03:29.005464 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:28.995767    2583 flags.go:64] FLAG: --tls-min-version=""
Jul 16 11:03:29.005464 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:28.995771    2583 flags.go:64] FLAG: --tls-private-key-file=""
Jul 16 11:03:29.005464 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:28.995774    2583 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 16 11:03:29.005464 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:28.995777    2583 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 16 11:03:29.005464 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:28.995780    2583 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 16 11:03:29.005464 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:28.995783    2583 flags.go:64] FLAG: --v="2"
Jul 16 11:03:29.005464 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:28.995787    2583 flags.go:64] FLAG: --version="false"
Jul 16 11:03:29.005464 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:28.995798    2583 flags.go:64] FLAG: --vmodule=""
Jul 16 11:03:29.006604 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:28.995803    2583 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 16 11:03:29.006604 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:28.995806    2583 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 16 11:03:29.006604 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.995935    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 16 11:03:29.006604 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.995940    2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 16 11:03:29.006604 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.995943    2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 16 11:03:29.006604 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.995946    2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 16 11:03:29.006604 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.995949    2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 16 11:03:29.006604 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.995952    2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 16 11:03:29.006604 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.995956    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 16 11:03:29.006604 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.995959    2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 16 11:03:29.006604 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.995963    2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 16 11:03:29.006604 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.995967    2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 16 11:03:29.006604 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.995971    2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 16 11:03:29.006604 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.995974    2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 16 11:03:29.006604 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.995979    2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 16 11:03:29.006604 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.995982    2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 16 11:03:29.006604 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.995985    2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 16 11:03:29.006604 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.995988    2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 16 11:03:29.006604 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.995991    2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 16 11:03:29.007173 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.995994    2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 16 11:03:29.007173 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.995997    2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 16 11:03:29.007173 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.996000    2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 16 11:03:29.007173 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.996003    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 16 11:03:29.007173 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.996006    2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 16 11:03:29.007173 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.996010    2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 16 11:03:29.007173 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.996013    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 16 11:03:29.007173 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.996016    2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 16 11:03:29.007173 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.996019    2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 16 11:03:29.007173 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.996022    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 16 11:03:29.007173 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.996025    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 16 11:03:29.007173 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.996028    2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 16 11:03:29.007173 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.996031    2583 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 16 11:03:29.007173 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.996034    2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 16 11:03:29.007173 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.996036    2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 16 11:03:29.007173 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.996039    2583 feature_gate.go:328] unrecognized feature gate: Example
Jul 16 11:03:29.007173 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.996042    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 16 11:03:29.007173 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.996045    2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 16 11:03:29.007173 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.996048    2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 16 11:03:29.007173 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.996050    2583 feature_gate.go:328] unrecognized feature gate: Example2
Jul 16 11:03:29.008052 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.996053    2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 16 11:03:29.008052 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.996056    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 16 11:03:29.008052 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.996059    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 16 11:03:29.008052 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.996061    2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 16 11:03:29.008052 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.996064    2583 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 16 11:03:29.008052 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.996067    2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 16 11:03:29.008052 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.996069    2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 16 11:03:29.008052 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.996074    2583 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 16 11:03:29.008052 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.996076    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 16 11:03:29.008052 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.996079    2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 16 11:03:29.008052 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.996082    2583 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 16 11:03:29.008052 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.996085    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 16 11:03:29.008052 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.996087    2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 16 11:03:29.008052 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.996090    2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 16 11:03:29.008052 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.996093    2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 16 11:03:29.008052 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.996096    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 16 11:03:29.008052 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.996099    2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 16 11:03:29.008052 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.996105    2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 16 11:03:29.008052 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.996108    2583 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 16 11:03:29.008052 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.996111    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 16 11:03:29.008609 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.996114    2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 16 11:03:29.008609 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.996117    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 16 11:03:29.008609 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.996119    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 16 11:03:29.008609 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.996122    2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 16 11:03:29.008609 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.996126    2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 16 11:03:29.008609 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.996130    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 16 11:03:29.008609 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.996134    2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 16 11:03:29.008609 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.996137    2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 16 11:03:29.008609 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.996140    2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 16 11:03:29.008609 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.996143    2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 16 11:03:29.008609 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.996146    2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 16 11:03:29.008609 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.996149    2583 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 16 11:03:29.008609 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.996153    2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 16 11:03:29.008609 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.996156    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 16 11:03:29.008609 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.996159    2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 16 11:03:29.008609 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.996162    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 16 11:03:29.008609 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.996165    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 16 11:03:29.008609 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.996168    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 16 11:03:29.008609 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.996171    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 16 11:03:29.009217 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.996175    2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 16 11:03:29.009217 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.996178    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 16 11:03:29.009217 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.996181    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 16 11:03:29.009217 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.996184    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 16 11:03:29.009217 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.996187    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 16 11:03:29.009217 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.996190    2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 16 11:03:29.009217 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.996193    2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 16 11:03:29.009217 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.996195    2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 16 11:03:29.009217 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.996198    2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 16 11:03:29.009217 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.996201    2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 16 11:03:29.009217 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.996206    2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 16 11:03:29.009217 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.996209    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 16 11:03:29.009217 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.996214    2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 16 11:03:29.009217 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.996218    2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 16 11:03:29.009217 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.996221    2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 16 11:03:29.009217 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.996224    2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 16 11:03:29.009217 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.996227    2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 16 11:03:29.009217 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.996230    2583 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 16 11:03:29.009217 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.996233    2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 16 11:03:29.009217 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.996235    2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 16 11:03:29.009762 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.996238    2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 16 11:03:29.009762 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.996241    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 16 11:03:29.009762 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:28.996244    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 16 11:03:29.009762 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:28.996250    2583 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 16 11:03:29.009762 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:29.004784    2583 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 16 11:03:29.009762 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:29.004811    2583 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 16 11:03:29.009762 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:29.004839    2583 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 16 11:03:29.009762 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:29.004848    2583 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 16 11:03:29.009762 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.004945    2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 16 11:03:29.009762 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.004954    2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 16 11:03:29.009762 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.004960    2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 16 11:03:29.009762 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.004966    2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 16 11:03:29.009762 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.004974    2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 16 11:03:29.009762 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.004980    2583 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 16 11:03:29.010165 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.004986    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 16 11:03:29.010165 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.004992    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 16 11:03:29.010165 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.004997    2583 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 16 11:03:29.010165 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.005003    2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 16 11:03:29.010165 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.005009    2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 16 11:03:29.010165 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.005014    2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 16 11:03:29.010165 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.005019    2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 16 11:03:29.010165 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.005024    2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 16 11:03:29.010165 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.005029    2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 16 11:03:29.010165 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.005034    2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 16 11:03:29.010165 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.005039    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 16 11:03:29.010165 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.005044    2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 16 11:03:29.010165 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.005048    2583 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 16 11:03:29.010165 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.005053    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 16 11:03:29.010165 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.005058    2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 16 11:03:29.010165 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.005063    2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 16 11:03:29.010165 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.005067    2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 16 11:03:29.010165 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.005072    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 16 11:03:29.010165 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.005078    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 16 11:03:29.010165 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.005083    2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 16 11:03:29.010724 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.005088    2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 16 11:03:29.010724 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.005092    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 16 11:03:29.010724 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.005098    2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 16 11:03:29.010724 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.005109    2583 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 16 11:03:29.010724 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.005114    2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 16 11:03:29.010724 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.005118    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 16 11:03:29.010724 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.005124    2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 16 11:03:29.010724 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.005129    2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 16 11:03:29.010724 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.005134    2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 16 11:03:29.010724 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.005138    2583 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 16 11:03:29.010724 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.005143    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 16 11:03:29.010724 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.005148    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 16 11:03:29.010724 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.005152    2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 16 11:03:29.010724 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.005157    2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 16 11:03:29.010724 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.005161    2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 16 11:03:29.010724 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.005167    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 16 11:03:29.010724 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.005171    2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 16 11:03:29.010724 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.005176    2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 16 11:03:29.010724 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.005181    2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 16 11:03:29.010724 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.005186    2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 16 11:03:29.011223 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.005190    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 16 11:03:29.011223 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.005195    2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 16 11:03:29.011223 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.005199    2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 16 11:03:29.011223 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.005204    2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 16 11:03:29.011223 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.005208    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 16 11:03:29.011223 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.005213    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 16 11:03:29.011223 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.005217    2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 16 11:03:29.011223 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.005221    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 16 11:03:29.011223 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.005226    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 16 11:03:29.011223 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.005230    2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 16 11:03:29.011223 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.005235    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 16 11:03:29.011223 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.005239    2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 16 11:03:29.011223 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.005244    2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 16 11:03:29.011223 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.005248    2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 16 11:03:29.011223 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.005254    2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 16 11:03:29.011223 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.005259    2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 16 11:03:29.011223 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.005265    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 16 11:03:29.011223 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.005272    2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 16 11:03:29.011223 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.005279    2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 16 11:03:29.011854 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.005284    2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 16 11:03:29.011854 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.005288    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 16 11:03:29.011854 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.005293    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 16 11:03:29.011854 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.005298    2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 16 11:03:29.011854 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.005303    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 16 11:03:29.011854 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.005307    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 16 11:03:29.011854 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.005312    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 16 11:03:29.011854 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.005317    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 16 11:03:29.011854 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.005321    2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 16 11:03:29.011854 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.005326    2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 16 11:03:29.011854 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.005330    2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 16 11:03:29.011854 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.005335    2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 16 11:03:29.011854 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.005342    2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 16 11:03:29.011854 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.005348    2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 16 11:03:29.011854 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.005353    2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 16 11:03:29.011854 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.005358    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 16 11:03:29.011854 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.005362    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 16 11:03:29.011854 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.005367    2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 16 11:03:29.011854 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.005371    2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 16 11:03:29.012346 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.005376    2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 16 11:03:29.012346 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.005380    2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 16 11:03:29.012346 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.005386    2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 16 11:03:29.012346 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.005391    2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 16 11:03:29.012346 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.005395    2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 16 11:03:29.012346 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.005400    2583 feature_gate.go:328] unrecognized feature gate: Example2
Jul 16 11:03:29.012346 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.005404    2583 feature_gate.go:328] unrecognized feature gate: Example
Jul 16 11:03:29.012346 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.005409    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 16 11:03:29.012346 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.005414    2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 16 11:03:29.012346 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.005419    2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 16 11:03:29.012346 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.005424    2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 16 11:03:29.012346 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.005429    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 16 11:03:29.012346 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.005434    2583 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 16 11:03:29.012346 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.005439    2583 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 16 11:03:29.012346 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.005444    2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 16 11:03:29.012860 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:29.005453    2583 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 16 11:03:29.012860 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.005660    2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 16 11:03:29.012860 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.005689    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 16 11:03:29.012860 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.005695    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 16 11:03:29.012860 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.005700    2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 16 11:03:29.012860 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.005704    2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 16 11:03:29.012860 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.005709    2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 16 11:03:29.012860 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.005714    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 16 11:03:29.012860 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.005719    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 16 11:03:29.012860 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.005724    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 16 11:03:29.012860 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.005729    2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 16 11:03:29.012860 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.005733    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 16 11:03:29.012860 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.005740    2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 16 11:03:29.012860 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.005746    2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 16 11:03:29.012860 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.005751    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 16 11:03:29.013255 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.005755    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 16 11:03:29.013255 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.005760    2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 16 11:03:29.013255 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.005765    2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 16 11:03:29.013255 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.005770    2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 16 11:03:29.013255 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.005775    2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 16 11:03:29.013255 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.005780    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 16 11:03:29.013255 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.005785    2583 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 16 11:03:29.013255 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.005789    2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 16 11:03:29.013255 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.005794    2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 16 11:03:29.013255 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.005798    2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 16 11:03:29.013255 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.005802    2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 16 11:03:29.013255 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.005807    2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 16 11:03:29.013255 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.005812    2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 16 11:03:29.013255 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.005818    2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 16 11:03:29.013255 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.005824    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 16 11:03:29.013255 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.005828    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 16 11:03:29.013255 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.005833    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 16 11:03:29.013255 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.005837    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 16 11:03:29.013255 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.005842    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 16 11:03:29.013255 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.005847    2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 16 11:03:29.014117 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.005852    2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 16 11:03:29.014117 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.005859    2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 16 11:03:29.014117 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.005866    2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 16 11:03:29.014117 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.005871    2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 16 11:03:29.014117 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.005876    2583 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 16 11:03:29.014117 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.005881    2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 16 11:03:29.014117 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.005886    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 16 11:03:29.014117 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.005890    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 16 11:03:29.014117 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.005894    2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 16 11:03:29.014117 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.005899    2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 16 11:03:29.014117 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.005903    2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 16 11:03:29.014117 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.005908    2583 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 16 11:03:29.014117 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.005920    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 16 11:03:29.014117 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.005925    2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 16 11:03:29.014117 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.005930    2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 16 11:03:29.014117 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.005935    2583 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 16 11:03:29.014117 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.005940    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 16 11:03:29.014117 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.005944    2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 16 11:03:29.014117 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.005949    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 16 11:03:29.014897 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.005953    2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 16 11:03:29.014897 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.005958    2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 16 11:03:29.014897 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.005963    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 16 11:03:29.014897 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.005967    2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 16 11:03:29.014897 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.005972    2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 16 11:03:29.014897 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.005977    2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 16 11:03:29.014897 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.005981    2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 16 11:03:29.014897 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.005986    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 16 11:03:29.014897 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.005992    2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 16 11:03:29.014897 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.005997    2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 16 11:03:29.014897 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.006001    2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 16 11:03:29.014897 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.006006    2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 16 11:03:29.014897 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.006010    2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 16 11:03:29.014897 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.006015    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 16 11:03:29.014897 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.006022    2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 16 11:03:29.014897 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.006029    2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 16 11:03:29.014897 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.006034    2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 16 11:03:29.014897 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.006040    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 16 11:03:29.014897 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.006045    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 16 11:03:29.015402 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.006050    2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 16 11:03:29.015402 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.006055    2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 16 11:03:29.015402 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.006059    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 16 11:03:29.015402 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.006065    2583 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 16 11:03:29.015402 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.006069    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 16 11:03:29.015402 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.006074    2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 16 11:03:29.015402 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.006078    2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 16 11:03:29.015402 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.006083    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 16 11:03:29.015402 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.006087    2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 16 11:03:29.015402 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.006091    2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 16 11:03:29.015402 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.006096    2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 16 11:03:29.015402 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.006101    2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 16 11:03:29.015402 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.006106    2583 feature_gate.go:328] unrecognized feature gate: Example
Jul 16 11:03:29.015402 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.006110    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 16 11:03:29.015402 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.006115    2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 16 11:03:29.015402 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.006120    2583 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 16 11:03:29.015402 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.006124    2583 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 16 11:03:29.015402 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.006129    2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 16 11:03:29.015402 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.006133    2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 16 11:03:29.015402 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.006138    2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 16 11:03:29.016233 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.006143    2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 16 11:03:29.016233 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.006147    2583 feature_gate.go:328] unrecognized feature gate: Example2
Jul 16 11:03:29.016233 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.006153    2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 16 11:03:29.016233 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.006157    2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 16 11:03:29.016233 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.006162    2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 16 11:03:29.016233 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.006167    2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 16 11:03:29.016233 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:29.006172    2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 16 11:03:29.016233 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:29.006180    2583 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 16 11:03:29.016233 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:29.006394    2583 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 16 11:03:29.016233 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:29.009804    2583 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 16 11:03:29.016233 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:29.011272    2583 server.go:1019] "Starting client certificate rotation"
Jul 16 11:03:29.016233 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:29.011404    2583 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 16 11:03:29.016233 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:29.011460    2583 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 16 11:03:29.016578 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:29.013953    2583 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 16 11:03:29.016956 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:29.016940    2583 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 16 11:03:29.031658 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:29.031627    2583 log.go:25] "Validated CRI v1 runtime API"
Jul 16 11:03:29.034204 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:29.034181    2583 log.go:25] "Validated CRI v1 image API"
Jul 16 11:03:29.035161 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:29.035144    2583 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 16 11:03:29.038843 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:29.038811    2583 fs.go:135] Filesystem UUIDs: map[2d9c30fd-15da-42da-a090-0a792d1dab74:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 fe9288eb-9d10-4770-97bd-46eb241d8e85:/dev/nvme0n1p3]
Jul 16 11:03:29.038918 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:29.038840    2583 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 16 11:03:29.046698 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:29.046545    2583 manager.go:216] Machine: {Timestamp:2026-07-16 11:03:29.044159042 +0000 UTC m=+1.678380490 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3032499 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec28d083e53003ebeb9a119c491bbbcd SystemUUID:ec28d083-e530-03eb-eb9a-119c491bbbcd BootID:248a9e27-0246-4ebf-af38-a79dce2fdee9 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:eb:93:42:8a:0d Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:eb:93:42:8a:0d Speed:0 Mtu:9001} {Name:ovs-system MacAddress:5a:77:2b:aa:fd:90 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 16 11:03:29.046698 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:29.046689    2583 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 16 11:03:29.046856 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:29.046832    2583 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 16 11:03:29.050959 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:29.050727    2583 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 16 11:03:29.051157 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:29.051138    2583 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 16 11:03:29.051194 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:29.050967    2583 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-129-28.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 16 11:03:29.052062 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:29.052049    2583 topology_manager.go:138] "Creating topology manager with none policy"
Jul 16 11:03:29.052100 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:29.052065    2583 container_manager_linux.go:309] "Creating device plugin manager"
Jul 16 11:03:29.052100 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:29.052074    2583 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 16 11:03:29.053917 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:29.053903    2583 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 16 11:03:29.054759 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:29.054749    2583 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 16 11:03:29.057980 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:29.057963    2583 state_mem.go:36] "Initialized new in-memory state store"
Jul 16 11:03:29.058128 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:29.058117    2583 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 16 11:03:29.058163 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:29.058152    2583 kubelet.go:486] "Attempting to sync node with API server"
Jul 16 11:03:29.058192 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:29.058165    2583 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 16 11:03:29.058192 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:29.058181    2583 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 16 11:03:29.058192 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:29.058191    2583 kubelet.go:398] "Adding apiserver pod source"
Jul 16 11:03:29.058294 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:29.058203    2583 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 16 11:03:29.058660 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:29.058644    2583 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 16 11:03:29.058761 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:29.058685    2583 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 16 11:03:29.061926 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:29.061899    2583 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 16 11:03:29.063439 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:29.063419    2583 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 16 11:03:29.063512 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:29.063454    2583 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 16 11:03:29.064089 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:29.064076    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 16 11:03:29.064137 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:29.064098    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 16 11:03:29.064137 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:29.064110    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 16 11:03:29.064137 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:29.064118    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 16 11:03:29.064137 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:29.064129    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 16 11:03:29.064137 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:29.064135    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 16 11:03:29.064323 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:29.064314    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 16 11:03:29.064358 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:29.064325    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 16 11:03:29.064358 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:29.064332    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 16 11:03:29.064358 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:29.064338    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 16 11:03:29.064358 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:29.064347    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 16 11:03:29.064358 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:29.064357    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 16 11:03:29.065315 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:29.065304    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 16 11:03:29.065351 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:29.065316    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 16 11:03:29.068878 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:29.068857    2583 server.go:1268] "Started kubelet"
Jul 16 11:03:29.069045 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:29.068970    2583 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 16 11:03:29.069115 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:29.069084    2583 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 16 11:03:29.069661 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:29.069590    2583 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 16 11:03:29.070014 ip-10-0-129-28 systemd[1]: Started Kubernetes Kubelet.
Jul 16 11:03:29.070889 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:29.070717    2583 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 16 11:03:29.072920 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:29.072903    2583 server.go:310] "Adding debug handlers to kubelet server"
Jul 16 11:03:29.073588 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:03:29.073553    2583 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 16 11:03:29.073770 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:29.073686    2583 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-129-28.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 16 11:03:29.073931 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:03:29.073879    2583 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-129-28.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 16 11:03:29.076818 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:29.076794    2583 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 16 11:03:29.076919 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:29.076817    2583 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 16 11:03:29.077533 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:03:29.077022    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-28.ec2.internal\" not found"
Jul 16 11:03:29.077770 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:29.077750    2583 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 16 11:03:29.078037 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:29.077753    2583 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 16 11:03:29.078037 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:29.077921    2583 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 16 11:03:29.078037 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:03:29.077951    2583 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 16 11:03:29.078186 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:29.078059    2583 reconstruct.go:97] "Volume reconstruction finished"
Jul 16 11:03:29.078186 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:29.078069    2583 reconciler.go:29] "Reconciler: start to sync state"
Jul 16 11:03:29.078300 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:29.078287    2583 factory.go:55] Registering systemd factory
Jul 16 11:03:29.078331 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:29.078305    2583 factory.go:223] Registration of the systemd container factory successfully
Jul 16 11:03:29.078551 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:29.078536    2583 factory.go:153] Registering CRI-O factory
Jul 16 11:03:29.078551 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:29.078551    2583 factory.go:223] Registration of the crio container factory successfully
Jul 16 11:03:29.078642 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:29.078598    2583 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 16 11:03:29.078707 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:29.078654    2583 factory.go:103] Registering Raw factory
Jul 16 11:03:29.078707 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:29.078667    2583 manager.go:1196] Started watching for new ooms in manager
Jul 16 11:03:29.079389 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:29.079375    2583 manager.go:349] Starting recovery of all containers
Jul 16 11:03:29.087492 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:03:29.087459    2583 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 16 11:03:29.087660 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:03:29.087547    2583 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-129-28.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 16 11:03:29.091332 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:03:29.087708    2583 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-28.ec2.internal.18c2c045f3ddf906  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-28.ec2.internal,UID:ip-10-0-129-28.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-129-28.ec2.internal,},FirstTimestamp:2026-07-16 11:03:29.068824838 +0000 UTC m=+1.703046285,LastTimestamp:2026-07-16 11:03:29.068824838 +0000 UTC m=+1.703046285,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-28.ec2.internal,}"
Jul 16 11:03:29.095518 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:29.095495    2583 manager.go:354] Recovery completed
Jul 16 11:03:29.100306 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:29.100289    2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 16 11:03:29.102372 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:29.102351    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-28.ec2.internal" event="NodeHasSufficientMemory"
Jul 16 11:03:29.102460 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:29.102388    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-28.ec2.internal" event="NodeHasNoDiskPressure"
Jul 16 11:03:29.102460 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:29.102401    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-28.ec2.internal" event="NodeHasSufficientPID"
Jul 16 11:03:29.102993 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:29.102977    2583 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 16 11:03:29.102993 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:29.102989    2583 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 16 11:03:29.103106 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:29.103007    2583 state_mem.go:36] "Initialized new in-memory state store"
Jul 16 11:03:29.103106 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:29.102988    2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 16 11:03:29.104170 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:29.104154    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-28.ec2.internal" event="NodeHasSufficientMemory"
Jul 16 11:03:29.104234 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:29.104180    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-28.ec2.internal" event="NodeHasNoDiskPressure"
Jul 16 11:03:29.104234 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:29.104193    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-28.ec2.internal" event="NodeHasSufficientPID"
Jul 16 11:03:29.104482 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:29.104472    2583 policy_none.go:49] "None policy: Start"
Jul 16 11:03:29.104529 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:29.104488    2583 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 16 11:03:29.104529 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:29.104500    2583 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 16 11:03:29.105142 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:03:29.105062    2583 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-28.ec2.internal.18c2c045f5ddd4ea  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-28.ec2.internal,UID:ip-10-0-129-28.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-129-28.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-129-28.ec2.internal,},FirstTimestamp:2026-07-16 11:03:29.102370026 +0000 UTC m=+1.736591475,LastTimestamp:2026-07-16 11:03:29.102370026 +0000 UTC m=+1.736591475,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-28.ec2.internal,}"
Jul 16 11:03:29.105673 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:29.105653    2583 policy_none.go:47] "Start"
Jul 16 11:03:29.122114 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:03:29.122000    2583 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-28.ec2.internal.18c2c045f5de3385  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-28.ec2.internal,UID:ip-10-0-129-28.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-129-28.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-129-28.ec2.internal,},FirstTimestamp:2026-07-16 11:03:29.102394245 +0000 UTC m=+1.736615692,LastTimestamp:2026-07-16 11:03:29.102394245 +0000 UTC m=+1.736615692,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-28.ec2.internal,}"
Jul 16 11:03:29.135896 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:03:29.135639    2583 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-28.ec2.internal.18c2c045f5de5e0a  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-28.ec2.internal,UID:ip-10-0-129-28.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-129-28.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-129-28.ec2.internal,},FirstTimestamp:2026-07-16 11:03:29.10240513 +0000 UTC m=+1.736626578,LastTimestamp:2026-07-16 11:03:29.10240513 +0000 UTC m=+1.736626578,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-28.ec2.internal,}"
Jul 16 11:03:29.147543 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:29.147524    2583 manager.go:337] "Starting Device Plugin manager"
Jul 16 11:03:29.147652 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:03:29.147574    2583 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 16 11:03:29.147652 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:29.147588    2583 server.go:87] "Starting device plugin registration server"
Jul 16 11:03:29.147923 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:29.147911    2583 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 16 11:03:29.147974 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:29.147925    2583 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 16 11:03:29.148048 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:29.148028    2583 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 16 11:03:29.148245 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:29.148181    2583 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 16 11:03:29.148245 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:29.148197    2583 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 16 11:03:29.148851 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:03:29.148826    2583 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 16 11:03:29.148952 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:03:29.148891    2583 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-129-28.ec2.internal\" not found"
Jul 16 11:03:29.150086 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:03:29.150009    2583 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-129-28.ec2.internal.18c2c045f5ddd4ea\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-28.ec2.internal.18c2c045f5ddd4ea  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-28.ec2.internal,UID:ip-10-0-129-28.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-129-28.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-129-28.ec2.internal,},FirstTimestamp:2026-07-16 11:03:29.102370026 +0000 UTC m=+1.736591475,LastTimestamp:2026-07-16 11:03:29.104167396 +0000 UTC m=+1.738388847,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-28.ec2.internal,}"
Jul 16 11:03:29.161988 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:03:29.161897    2583 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-129-28.ec2.internal.18c2c045f5de3385\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-28.ec2.internal.18c2c045f5de3385  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-28.ec2.internal,UID:ip-10-0-129-28.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-129-28.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-129-28.ec2.internal,},FirstTimestamp:2026-07-16 11:03:29.102394245 +0000 UTC m=+1.736615692,LastTimestamp:2026-07-16 11:03:29.104185689 +0000 UTC m=+1.738407138,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-28.ec2.internal,}"
Jul 16 11:03:29.165234 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:29.165212    2583 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-2zqq9"
Jul 16 11:03:29.172331 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:03:29.172243    2583 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-129-28.ec2.internal.18c2c045f5de5e0a\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-28.ec2.internal.18c2c045f5de5e0a  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-28.ec2.internal,UID:ip-10-0-129-28.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-129-28.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-129-28.ec2.internal,},FirstTimestamp:2026-07-16 11:03:29.10240513 +0000 UTC m=+1.736626578,LastTimestamp:2026-07-16 11:03:29.104200002 +0000 UTC m=+1.738421451,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-28.ec2.internal,}"
Jul 16 11:03:29.187309 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:29.180560    2583 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-2zqq9"
Jul 16 11:03:29.187309 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:03:29.182024    2583 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-28.ec2.internal.18c2c045f8b84ef1  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-28.ec2.internal,UID:ip-10-0-129-28.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeAllocatableEnforced,Message:Updated Node Allocatable limit across pods,Source:EventSource{Component:kubelet,Host:ip-10-0-129-28.ec2.internal,},FirstTimestamp:2026-07-16 11:03:29.150242545 +0000 UTC m=+1.784463982,LastTimestamp:2026-07-16 11:03:29.150242545 +0000 UTC m=+1.784463982,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-28.ec2.internal,}"
Jul 16 11:03:29.220788 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:29.220753    2583 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 16 11:03:29.222189 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:29.222162    2583 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 16 11:03:29.222285 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:29.222221    2583 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 16 11:03:29.222285 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:29.222253    2583 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 16 11:03:29.222389 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:03:29.222300    2583 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 16 11:03:29.232891 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:29.232862    2583 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 16 11:03:29.248797 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:29.248727    2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 16 11:03:29.249775 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:29.249749    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-28.ec2.internal" event="NodeHasSufficientMemory"
Jul 16 11:03:29.249859 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:29.249788    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-28.ec2.internal" event="NodeHasNoDiskPressure"
Jul 16 11:03:29.249859 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:29.249802    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-28.ec2.internal" event="NodeHasSufficientPID"
Jul 16 11:03:29.249859 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:29.249830    2583 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-129-28.ec2.internal"
Jul 16 11:03:29.261546 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:29.261512    2583 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-129-28.ec2.internal"
Jul 16 11:03:29.261739 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:03:29.261548    2583 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-129-28.ec2.internal\": node \"ip-10-0-129-28.ec2.internal\" not found"
Jul 16 11:03:29.286989 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:03:29.286950    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-28.ec2.internal\" not found"
Jul 16 11:03:29.323150 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:29.323092    2583 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-28.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-129-28.ec2.internal"]
Jul 16 11:03:29.323304 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:29.323196    2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 16 11:03:29.324224 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:29.324205    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-28.ec2.internal" event="NodeHasSufficientMemory"
Jul 16 11:03:29.324323 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:29.324237    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-28.ec2.internal" event="NodeHasNoDiskPressure"
Jul 16 11:03:29.324323 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:29.324248    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-28.ec2.internal" event="NodeHasSufficientPID"
Jul 16 11:03:29.325683 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:29.325661    2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 16 11:03:29.325804 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:29.325776    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-28.ec2.internal"
Jul 16 11:03:29.325862 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:29.325812    2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 16 11:03:29.326517 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:29.326498    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-28.ec2.internal" event="NodeHasSufficientMemory"
Jul 16 11:03:29.326609 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:29.326529    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-28.ec2.internal" event="NodeHasNoDiskPressure"
Jul 16 11:03:29.326609 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:29.326540    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-28.ec2.internal" event="NodeHasSufficientPID"
Jul 16 11:03:29.326609 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:29.326546    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-28.ec2.internal" event="NodeHasSufficientMemory"
Jul 16 11:03:29.326609 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:29.326569    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-28.ec2.internal" event="NodeHasNoDiskPressure"
Jul 16 11:03:29.326609 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:29.326599    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-28.ec2.internal" event="NodeHasSufficientPID"
Jul 16 11:03:29.327990 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:29.327976    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-28.ec2.internal"
Jul 16 11:03:29.328046 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:29.328007    2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 16 11:03:29.328870 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:29.328852    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-28.ec2.internal" event="NodeHasSufficientMemory"
Jul 16 11:03:29.328957 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:29.328884    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-28.ec2.internal" event="NodeHasNoDiskPressure"
Jul 16 11:03:29.328957 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:29.328901    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-28.ec2.internal" event="NodeHasSufficientPID"
Jul 16 11:03:29.347447 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:03:29.347408    2583 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-28.ec2.internal\" not found" node="ip-10-0-129-28.ec2.internal"
Jul 16 11:03:29.352702 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:03:29.352667    2583 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-28.ec2.internal\" not found" node="ip-10-0-129-28.ec2.internal"
Jul 16 11:03:29.379229 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:29.379200    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/b31dba48238fdc286bfab6bd4db2f24f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-28.ec2.internal\" (UID: \"b31dba48238fdc286bfab6bd4db2f24f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-28.ec2.internal"
Jul 16 11:03:29.379375 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:29.379246    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b31dba48238fdc286bfab6bd4db2f24f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-28.ec2.internal\" (UID: \"b31dba48238fdc286bfab6bd4db2f24f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-28.ec2.internal"
Jul 16 11:03:29.379375 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:29.379289    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/43ba1ebae5d7ee21a4751f58c11fde6b-config\") pod \"kube-apiserver-proxy-ip-10-0-129-28.ec2.internal\" (UID: \"43ba1ebae5d7ee21a4751f58c11fde6b\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-28.ec2.internal"
Jul 16 11:03:29.387482 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:03:29.387448    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-28.ec2.internal\" not found"
Jul 16 11:03:29.479958 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:29.479922    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b31dba48238fdc286bfab6bd4db2f24f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-28.ec2.internal\" (UID: \"b31dba48238fdc286bfab6bd4db2f24f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-28.ec2.internal"
Jul 16 11:03:29.480018 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:29.479962    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/43ba1ebae5d7ee21a4751f58c11fde6b-config\") pod \"kube-apiserver-proxy-ip-10-0-129-28.ec2.internal\" (UID: \"43ba1ebae5d7ee21a4751f58c11fde6b\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-28.ec2.internal"
Jul 16 11:03:29.480018 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:29.479980    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/b31dba48238fdc286bfab6bd4db2f24f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-28.ec2.internal\" (UID: \"b31dba48238fdc286bfab6bd4db2f24f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-28.ec2.internal"
Jul 16 11:03:29.480082 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:29.480031    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b31dba48238fdc286bfab6bd4db2f24f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-28.ec2.internal\" (UID: \"b31dba48238fdc286bfab6bd4db2f24f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-28.ec2.internal"
Jul 16 11:03:29.480116 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:29.480098    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/b31dba48238fdc286bfab6bd4db2f24f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-28.ec2.internal\" (UID: \"b31dba48238fdc286bfab6bd4db2f24f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-28.ec2.internal"
Jul 16 11:03:29.480197 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:29.480123    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/43ba1ebae5d7ee21a4751f58c11fde6b-config\") pod \"kube-apiserver-proxy-ip-10-0-129-28.ec2.internal\" (UID: \"43ba1ebae5d7ee21a4751f58c11fde6b\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-28.ec2.internal"
Jul 16 11:03:29.487844 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:03:29.487811    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-28.ec2.internal\" not found"
Jul 16 11:03:29.588867 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:03:29.588798    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-28.ec2.internal\" not found"
Jul 16 11:03:29.652999 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:29.652953    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-28.ec2.internal"
Jul 16 11:03:29.655640 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:29.655618    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-28.ec2.internal"
Jul 16 11:03:29.689660 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:03:29.689621    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-28.ec2.internal\" not found"
Jul 16 11:03:29.790154 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:03:29.790103    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-28.ec2.internal\" not found"
Jul 16 11:03:29.891036 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:03:29.890945    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-28.ec2.internal\" not found"
Jul 16 11:03:29.962153 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:29.962122    2583 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 16 11:03:29.991593 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:03:29.991551    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-28.ec2.internal\" not found"
Jul 16 11:03:30.011203 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:30.011166    2583 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 16 11:03:30.011381 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:30.011360    2583 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 16 11:03:30.011426 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:30.011384    2583 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 16 11:03:30.092063 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:03:30.092016    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-28.ec2.internal\" not found"
Jul 16 11:03:30.148868 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:30.148779    2583 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 16 11:03:30.179005 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:30.178975    2583 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-28.ec2.internal"
Jul 16 11:03:30.186771 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:30.186736    2583 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-15 10:58:29 +0000 UTC" deadline="2028-04-16 21:17:54.5771603 +0000 UTC"
Jul 16 11:03:30.186771 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:30.186768    2583 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15370h14m24.390396061s"
Jul 16 11:03:30.198386 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:30.198361    2583 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 16 11:03:30.199822 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:30.199805    2583 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-28.ec2.internal"
Jul 16 11:03:30.209025 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:30.208998    2583 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 16 11:03:30.329696 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:30.329658    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb31dba48238fdc286bfab6bd4db2f24f.slice/crio-ddb629d3e4f03afa0cb14e1f89f409e76c74fc8a8a322c79ef771feb2ec2dc72 WatchSource:0}: Error finding container ddb629d3e4f03afa0cb14e1f89f409e76c74fc8a8a322c79ef771feb2ec2dc72: Status 404 returned error can't find the container with id ddb629d3e4f03afa0cb14e1f89f409e76c74fc8a8a322c79ef771feb2ec2dc72
Jul 16 11:03:30.336818 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:30.336798    2583 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 16 11:03:30.413038 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:30.412946    2583 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 16 11:03:31.061370 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.061317    2583 apiserver.go:52] "Watching apiserver"
Jul 16 11:03:31.069686 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.069627    2583 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 16 11:03:31.070166 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.070135    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-node-tuning-operator/tuned-cr2x7","openshift-dns/node-resolver-z5dtv","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-28.ec2.internal","openshift-multus/network-metrics-daemon-qcl5d","openshift-network-operator/iptables-alerter-x6lfk","kube-system/konnectivity-agent-qxpqt","kube-system/kube-apiserver-proxy-ip-10-0-129-28.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wtqvh","openshift-image-registry/node-ca-6xfs6","openshift-multus/multus-additional-cni-plugins-8r89z","openshift-multus/multus-zs9vs","openshift-network-diagnostics/network-check-target-rc7w4","openshift-ovn-kubernetes/ovnkube-node-2db6t","kube-system/global-pull-secret-syncer-hlm2s"]
Jul 16 11:03:31.072263 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.072239    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wtqvh"
Jul 16 11:03:31.073479 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.073453    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-zs9vs"
Jul 16 11:03:31.074975 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.074949    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 16 11:03:31.075067 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.075009    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 16 11:03:31.075067 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.075046    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-dzhr8\""
Jul 16 11:03:31.075067 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.075063    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 16 11:03:31.075392 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.075377    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qcl5d"
Jul 16 11:03:31.075649 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:03:31.075453    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qcl5d" podUID="d4e7c597-9836-4a29-8591-e56938995710"
Jul 16 11:03:31.075884 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.075861    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 16 11:03:31.076273 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.076095    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-blj22\""
Jul 16 11:03:31.076273 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.076177    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 16 11:03:31.076273 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.076197    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 16 11:03:31.076485 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.076461    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 16 11:03:31.076848 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.076828    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-x6lfk"
Jul 16 11:03:31.077256 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.077231    2583 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 16 11:03:31.078431 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.078406    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-qxpqt"
Jul 16 11:03:31.079310 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.079291    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 16 11:03:31.079310 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.079300    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 16 11:03:31.079465 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.079374    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-xbjqj\""
Jul 16 11:03:31.079526 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.079462    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 16 11:03:31.079983 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.079962    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-cr2x7"
Jul 16 11:03:31.080935 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.080916    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 16 11:03:31.081251 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.081213    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-8nbfp\""
Jul 16 11:03:31.081687 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.081435    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-6xfs6"
Jul 16 11:03:31.081687 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.081588    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 16 11:03:31.082229 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.082210    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 16 11:03:31.082480 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.082462    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-zkg8w\""
Jul 16 11:03:31.082573 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.082461    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 16 11:03:31.083290 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.083270    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-8r89z"
Jul 16 11:03:31.084361 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.084264    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 16 11:03:31.084361 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.084301    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 16 11:03:31.084811 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.084787    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-2sjgd\""
Jul 16 11:03:31.085358 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.085263    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 16 11:03:31.088016 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.085631    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-z5dtv"
Jul 16 11:03:31.088016 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.086692    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-4wd59\""
Jul 16 11:03:31.088178 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.088073    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rc7w4"
Jul 16 11:03:31.088178 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.088120    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 16 11:03:31.088285 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:03:31.088181    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rc7w4" podUID="3d0ee8e8-4018-4963-b553-af9b6ecb0f80"
Jul 16 11:03:31.088820 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.088557    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-tkdh6\""
Jul 16 11:03:31.088820 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.088596    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 16 11:03:31.088820 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.088648    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 16 11:03:31.088820 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.088606    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 16 11:03:31.089356 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.089333    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5baff3f6-85da-46fb-b42b-2e70e7d77cc4-multus-conf-dir\") pod \"multus-zs9vs\" (UID: \"5baff3f6-85da-46fb-b42b-2e70e7d77cc4\") " pod="openshift-multus/multus-zs9vs"
Jul 16 11:03:31.089452 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.089372    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8rbc4\" (UniqueName: \"kubernetes.io/projected/2d4103af-f06a-4592-83e0-8625e6267267-kube-api-access-8rbc4\") pod \"iptables-alerter-x6lfk\" (UID: \"2d4103af-f06a-4592-83e0-8625e6267267\") " pod="openshift-network-operator/iptables-alerter-x6lfk"
Jul 16 11:03:31.089452 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.089413    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/c24aaddd-b44c-4155-a7c5-fadb281f9b39-konnectivity-ca\") pod \"konnectivity-agent-qxpqt\" (UID: \"c24aaddd-b44c-4155-a7c5-fadb281f9b39\") " pod="kube-system/konnectivity-agent-qxpqt"
Jul 16 11:03:31.089552 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.089513    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/21e8d664-4b0d-45cb-914f-7f55bb26d1f0-socket-dir\") pod \"aws-ebs-csi-driver-node-wtqvh\" (UID: \"21e8d664-4b0d-45cb-914f-7f55bb26d1f0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wtqvh"
Jul 16 11:03:31.089599 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.089557    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/21e8d664-4b0d-45cb-914f-7f55bb26d1f0-etc-selinux\") pod \"aws-ebs-csi-driver-node-wtqvh\" (UID: \"21e8d664-4b0d-45cb-914f-7f55bb26d1f0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wtqvh"
Jul 16 11:03:31.089599 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.089592    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d4e7c597-9836-4a29-8591-e56938995710-metrics-certs\") pod \"network-metrics-daemon-qcl5d\" (UID: \"d4e7c597-9836-4a29-8591-e56938995710\") " pod="openshift-multus/network-metrics-daemon-qcl5d"
Jul 16 11:03:31.089731 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.089618    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5baff3f6-85da-46fb-b42b-2e70e7d77cc4-system-cni-dir\") pod \"multus-zs9vs\" (UID: \"5baff3f6-85da-46fb-b42b-2e70e7d77cc4\") " pod="openshift-multus/multus-zs9vs"
Jul 16 11:03:31.089731 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.089660    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5baff3f6-85da-46fb-b42b-2e70e7d77cc4-cnibin\") pod \"multus-zs9vs\" (UID: \"5baff3f6-85da-46fb-b42b-2e70e7d77cc4\") " pod="openshift-multus/multus-zs9vs"
Jul 16 11:03:31.089731 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.089683    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5baff3f6-85da-46fb-b42b-2e70e7d77cc4-host-var-lib-cni-bin\") pod \"multus-zs9vs\" (UID: \"5baff3f6-85da-46fb-b42b-2e70e7d77cc4\") " pod="openshift-multus/multus-zs9vs"
Jul 16 11:03:31.089731 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.089708    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/5baff3f6-85da-46fb-b42b-2e70e7d77cc4-host-var-lib-cni-multus\") pod \"multus-zs9vs\" (UID: \"5baff3f6-85da-46fb-b42b-2e70e7d77cc4\") " pod="openshift-multus/multus-zs9vs"
Jul 16 11:03:31.089903 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.089735    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/5baff3f6-85da-46fb-b42b-2e70e7d77cc4-hostroot\") pod \"multus-zs9vs\" (UID: \"5baff3f6-85da-46fb-b42b-2e70e7d77cc4\") " pod="openshift-multus/multus-zs9vs"
Jul 16 11:03:31.089903 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.089767    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/5baff3f6-85da-46fb-b42b-2e70e7d77cc4-multus-socket-dir-parent\") pod \"multus-zs9vs\" (UID: \"5baff3f6-85da-46fb-b42b-2e70e7d77cc4\") " pod="openshift-multus/multus-zs9vs"
Jul 16 11:03:31.089903 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.089793    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-85x99\" (UniqueName: \"kubernetes.io/projected/5baff3f6-85da-46fb-b42b-2e70e7d77cc4-kube-api-access-85x99\") pod \"multus-zs9vs\" (UID: \"5baff3f6-85da-46fb-b42b-2e70e7d77cc4\") " pod="openshift-multus/multus-zs9vs"
Jul 16 11:03:31.089903 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.089820    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/2d4103af-f06a-4592-83e0-8625e6267267-iptables-alerter-script\") pod \"iptables-alerter-x6lfk\" (UID: \"2d4103af-f06a-4592-83e0-8625e6267267\") " pod="openshift-network-operator/iptables-alerter-x6lfk"
Jul 16 11:03:31.089903 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.089853    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/c24aaddd-b44c-4155-a7c5-fadb281f9b39-agent-certs\") pod \"konnectivity-agent-qxpqt\" (UID: \"c24aaddd-b44c-4155-a7c5-fadb281f9b39\") " pod="kube-system/konnectivity-agent-qxpqt"
Jul 16 11:03:31.089903 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.089876    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/21e8d664-4b0d-45cb-914f-7f55bb26d1f0-device-dir\") pod \"aws-ebs-csi-driver-node-wtqvh\" (UID: \"21e8d664-4b0d-45cb-914f-7f55bb26d1f0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wtqvh"
Jul 16 11:03:31.090188 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.089922    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d8xds\" (UniqueName: \"kubernetes.io/projected/21e8d664-4b0d-45cb-914f-7f55bb26d1f0-kube-api-access-d8xds\") pod \"aws-ebs-csi-driver-node-wtqvh\" (UID: \"21e8d664-4b0d-45cb-914f-7f55bb26d1f0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wtqvh"
Jul 16 11:03:31.090188 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.089977    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5baff3f6-85da-46fb-b42b-2e70e7d77cc4-os-release\") pod \"multus-zs9vs\" (UID: \"5baff3f6-85da-46fb-b42b-2e70e7d77cc4\") " pod="openshift-multus/multus-zs9vs"
Jul 16 11:03:31.090188 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.090005    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/5baff3f6-85da-46fb-b42b-2e70e7d77cc4-multus-daemon-config\") pod \"multus-zs9vs\" (UID: \"5baff3f6-85da-46fb-b42b-2e70e7d77cc4\") " pod="openshift-multus/multus-zs9vs"
Jul 16 11:03:31.090188 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.090029    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/5baff3f6-85da-46fb-b42b-2e70e7d77cc4-host-run-multus-certs\") pod \"multus-zs9vs\" (UID: \"5baff3f6-85da-46fb-b42b-2e70e7d77cc4\") " pod="openshift-multus/multus-zs9vs"
Jul 16 11:03:31.090188 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.090057    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/21e8d664-4b0d-45cb-914f-7f55bb26d1f0-kubelet-dir\") pod \"aws-ebs-csi-driver-node-wtqvh\" (UID: \"21e8d664-4b0d-45cb-914f-7f55bb26d1f0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wtqvh"
Jul 16 11:03:31.090188 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.090089    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/21e8d664-4b0d-45cb-914f-7f55bb26d1f0-registration-dir\") pod \"aws-ebs-csi-driver-node-wtqvh\" (UID: \"21e8d664-4b0d-45cb-914f-7f55bb26d1f0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wtqvh"
Jul 16 11:03:31.090188 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.090125    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/21e8d664-4b0d-45cb-914f-7f55bb26d1f0-sys-fs\") pod \"aws-ebs-csi-driver-node-wtqvh\" (UID: \"21e8d664-4b0d-45cb-914f-7f55bb26d1f0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wtqvh"
Jul 16 11:03:31.090188 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.090167    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5baff3f6-85da-46fb-b42b-2e70e7d77cc4-cni-binary-copy\") pod \"multus-zs9vs\" (UID: \"5baff3f6-85da-46fb-b42b-2e70e7d77cc4\") " pod="openshift-multus/multus-zs9vs"
Jul 16 11:03:31.090547 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.090205    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5baff3f6-85da-46fb-b42b-2e70e7d77cc4-host-run-netns\") pod \"multus-zs9vs\" (UID: \"5baff3f6-85da-46fb-b42b-2e70e7d77cc4\") " pod="openshift-multus/multus-zs9vs"
Jul 16 11:03:31.090547 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.090231    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5baff3f6-85da-46fb-b42b-2e70e7d77cc4-host-var-lib-kubelet\") pod \"multus-zs9vs\" (UID: \"5baff3f6-85da-46fb-b42b-2e70e7d77cc4\") " pod="openshift-multus/multus-zs9vs"
Jul 16 11:03:31.090547 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.090268    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2d4103af-f06a-4592-83e0-8625e6267267-host-slash\") pod \"iptables-alerter-x6lfk\" (UID: \"2d4103af-f06a-4592-83e0-8625e6267267\") " pod="openshift-network-operator/iptables-alerter-x6lfk"
Jul 16 11:03:31.090547 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.090291    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5baff3f6-85da-46fb-b42b-2e70e7d77cc4-multus-cni-dir\") pod \"multus-zs9vs\" (UID: \"5baff3f6-85da-46fb-b42b-2e70e7d77cc4\") " pod="openshift-multus/multus-zs9vs"
Jul 16 11:03:31.090547 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.090308    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h5wj2\" (UniqueName: \"kubernetes.io/projected/d4e7c597-9836-4a29-8591-e56938995710-kube-api-access-h5wj2\") pod \"network-metrics-daemon-qcl5d\" (UID: \"d4e7c597-9836-4a29-8591-e56938995710\") " pod="openshift-multus/network-metrics-daemon-qcl5d"
Jul 16 11:03:31.090547 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.090352    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/5baff3f6-85da-46fb-b42b-2e70e7d77cc4-host-run-k8s-cni-cncf-io\") pod \"multus-zs9vs\" (UID: \"5baff3f6-85da-46fb-b42b-2e70e7d77cc4\") " pod="openshift-multus/multus-zs9vs"
Jul 16 11:03:31.090547 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.090401    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5baff3f6-85da-46fb-b42b-2e70e7d77cc4-etc-kubernetes\") pod \"multus-zs9vs\" (UID: \"5baff3f6-85da-46fb-b42b-2e70e7d77cc4\") " pod="openshift-multus/multus-zs9vs"
Jul 16 11:03:31.090892 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.090837    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-2db6t"
Jul 16 11:03:31.092443 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.092422    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hlm2s"
Jul 16 11:03:31.092531 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:03:31.092497    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hlm2s" podUID="c88b6ce0-4926-4de9-942e-7f1b30c33e9e"
Jul 16 11:03:31.093266 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.093238    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-nbc8v\""
Jul 16 11:03:31.094167 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.094123    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 16 11:03:31.094292 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.094273    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 16 11:03:31.094990 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.094969    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 16 11:03:31.094990 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.094979    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 16 11:03:31.095133 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.095015    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 16 11:03:31.095133 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.095068    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 16 11:03:31.111828 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.111800    2583 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 16 11:03:31.163303 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.163269    2583 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 16 11:03:31.179447 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.179414    2583 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 16 11:03:31.180743 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.180714    2583 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-m7gbm"
Jul 16 11:03:31.191156 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.191118    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/a3d44785-92d8-4a5e-bc2e-b91f4fa8b509-hosts-file\") pod \"node-resolver-z5dtv\" (UID: \"a3d44785-92d8-4a5e-bc2e-b91f4fa8b509\") " pod="openshift-dns/node-resolver-z5dtv"
Jul 16 11:03:31.191338 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.191186    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/c24aaddd-b44c-4155-a7c5-fadb281f9b39-konnectivity-ca\") pod \"konnectivity-agent-qxpqt\" (UID: \"c24aaddd-b44c-4155-a7c5-fadb281f9b39\") " pod="kube-system/konnectivity-agent-qxpqt"
Jul 16 11:03:31.191338 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.191214    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8de3fcaf-5b76-4d0f-8453-60b892832242-etc-sysctl-conf\") pod \"tuned-cr2x7\" (UID: \"8de3fcaf-5b76-4d0f-8453-60b892832242\") " pod="openshift-cluster-node-tuning-operator/tuned-cr2x7"
Jul 16 11:03:31.191338 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.191247    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5baff3f6-85da-46fb-b42b-2e70e7d77cc4-system-cni-dir\") pod \"multus-zs9vs\" (UID: \"5baff3f6-85da-46fb-b42b-2e70e7d77cc4\") " pod="openshift-multus/multus-zs9vs"
Jul 16 11:03:31.191338 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.191280    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5baff3f6-85da-46fb-b42b-2e70e7d77cc4-cnibin\") pod \"multus-zs9vs\" (UID: \"5baff3f6-85da-46fb-b42b-2e70e7d77cc4\") " pod="openshift-multus/multus-zs9vs"
Jul 16 11:03:31.191338 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.191304    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/5baff3f6-85da-46fb-b42b-2e70e7d77cc4-host-var-lib-cni-multus\") pod \"multus-zs9vs\" (UID: \"5baff3f6-85da-46fb-b42b-2e70e7d77cc4\") " pod="openshift-multus/multus-zs9vs"
Jul 16 11:03:31.191540 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.191408    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/5baff3f6-85da-46fb-b42b-2e70e7d77cc4-hostroot\") pod \"multus-zs9vs\" (UID: \"5baff3f6-85da-46fb-b42b-2e70e7d77cc4\") " pod="openshift-multus/multus-zs9vs"
Jul 16 11:03:31.191540 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.191411    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5baff3f6-85da-46fb-b42b-2e70e7d77cc4-system-cni-dir\") pod \"multus-zs9vs\" (UID: \"5baff3f6-85da-46fb-b42b-2e70e7d77cc4\") " pod="openshift-multus/multus-zs9vs"
Jul 16 11:03:31.191540 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.191432    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/5baff3f6-85da-46fb-b42b-2e70e7d77cc4-multus-socket-dir-parent\") pod \"multus-zs9vs\" (UID: \"5baff3f6-85da-46fb-b42b-2e70e7d77cc4\") " pod="openshift-multus/multus-zs9vs"
Jul 16 11:03:31.191540 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.191478    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/2d4103af-f06a-4592-83e0-8625e6267267-iptables-alerter-script\") pod \"iptables-alerter-x6lfk\" (UID: \"2d4103af-f06a-4592-83e0-8625e6267267\") " pod="openshift-network-operator/iptables-alerter-x6lfk"
Jul 16 11:03:31.191540 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.191512    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8de3fcaf-5b76-4d0f-8453-60b892832242-etc-sysconfig\") pod \"tuned-cr2x7\" (UID: \"8de3fcaf-5b76-4d0f-8453-60b892832242\") " pod="openshift-cluster-node-tuning-operator/tuned-cr2x7"
Jul 16 11:03:31.191813 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.191544    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/c24aaddd-b44c-4155-a7c5-fadb281f9b39-agent-certs\") pod \"konnectivity-agent-qxpqt\" (UID: \"c24aaddd-b44c-4155-a7c5-fadb281f9b39\") " pod="kube-system/konnectivity-agent-qxpqt"
Jul 16 11:03:31.191813 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.191553    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/5baff3f6-85da-46fb-b42b-2e70e7d77cc4-host-var-lib-cni-multus\") pod \"multus-zs9vs\" (UID: \"5baff3f6-85da-46fb-b42b-2e70e7d77cc4\") " pod="openshift-multus/multus-zs9vs"
Jul 16 11:03:31.191813 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.191587    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/5baff3f6-85da-46fb-b42b-2e70e7d77cc4-multus-socket-dir-parent\") pod \"multus-zs9vs\" (UID: \"5baff3f6-85da-46fb-b42b-2e70e7d77cc4\") " pod="openshift-multus/multus-zs9vs"
Jul 16 11:03:31.191813 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.191574    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5baff3f6-85da-46fb-b42b-2e70e7d77cc4-os-release\") pod \"multus-zs9vs\" (UID: \"5baff3f6-85da-46fb-b42b-2e70e7d77cc4\") " pod="openshift-multus/multus-zs9vs"
Jul 16 11:03:31.191813 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.191644    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/5baff3f6-85da-46fb-b42b-2e70e7d77cc4-hostroot\") pod \"multus-zs9vs\" (UID: \"5baff3f6-85da-46fb-b42b-2e70e7d77cc4\") " pod="openshift-multus/multus-zs9vs"
Jul 16 11:03:31.191813 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.191659    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5baff3f6-85da-46fb-b42b-2e70e7d77cc4-os-release\") pod \"multus-zs9vs\" (UID: \"5baff3f6-85da-46fb-b42b-2e70e7d77cc4\") " pod="openshift-multus/multus-zs9vs"
Jul 16 11:03:31.191813 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.191653    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2d4103af-f06a-4592-83e0-8625e6267267-host-slash\") pod \"iptables-alerter-x6lfk\" (UID: \"2d4103af-f06a-4592-83e0-8625e6267267\") " pod="openshift-network-operator/iptables-alerter-x6lfk"
Jul 16 11:03:31.191813 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.191698    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2d4103af-f06a-4592-83e0-8625e6267267-host-slash\") pod \"iptables-alerter-x6lfk\" (UID: \"2d4103af-f06a-4592-83e0-8625e6267267\") " pod="openshift-network-operator/iptables-alerter-x6lfk"
Jul 16 11:03:31.191813 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.191732    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5baff3f6-85da-46fb-b42b-2e70e7d77cc4-cnibin\") pod \"multus-zs9vs\" (UID: \"5baff3f6-85da-46fb-b42b-2e70e7d77cc4\") " pod="openshift-multus/multus-zs9vs"
Jul 16 11:03:31.192249 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.192002    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8de3fcaf-5b76-4d0f-8453-60b892832242-etc-modprobe-d\") pod \"tuned-cr2x7\" (UID: \"8de3fcaf-5b76-4d0f-8453-60b892832242\") " pod="openshift-cluster-node-tuning-operator/tuned-cr2x7"
Jul 16 11:03:31.192249 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.192078    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8de3fcaf-5b76-4d0f-8453-60b892832242-lib-modules\") pod \"tuned-cr2x7\" (UID: \"8de3fcaf-5b76-4d0f-8453-60b892832242\") " pod="openshift-cluster-node-tuning-operator/tuned-cr2x7"
Jul 16 11:03:31.192249 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.192119    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/6fd1e802-78a8-4991-99ee-47cfea84ab62-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-8r89z\" (UID: \"6fd1e802-78a8-4991-99ee-47cfea84ab62\") " pod="openshift-multus/multus-additional-cni-plugins-8r89z"
Jul 16 11:03:31.192249 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.192130    2583 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 16 11:03:31.192249 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.192155    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6fd1e802-78a8-4991-99ee-47cfea84ab62-cni-binary-copy\") pod \"multus-additional-cni-plugins-8r89z\" (UID: \"6fd1e802-78a8-4991-99ee-47cfea84ab62\") " pod="openshift-multus/multus-additional-cni-plugins-8r89z"
Jul 16 11:03:31.192249 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.192182    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/21e8d664-4b0d-45cb-914f-7f55bb26d1f0-kubelet-dir\") pod \"aws-ebs-csi-driver-node-wtqvh\" (UID: \"21e8d664-4b0d-45cb-914f-7f55bb26d1f0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wtqvh"
Jul 16 11:03:31.192249 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.192214    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/21e8d664-4b0d-45cb-914f-7f55bb26d1f0-registration-dir\") pod \"aws-ebs-csi-driver-node-wtqvh\" (UID: \"21e8d664-4b0d-45cb-914f-7f55bb26d1f0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wtqvh"
Jul 16 11:03:31.192249 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.192243    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5baff3f6-85da-46fb-b42b-2e70e7d77cc4-cni-binary-copy\") pod \"multus-zs9vs\" (UID: \"5baff3f6-85da-46fb-b42b-2e70e7d77cc4\") " pod="openshift-multus/multus-zs9vs"
Jul 16 11:03:31.192720 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.192269    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/21e8d664-4b0d-45cb-914f-7f55bb26d1f0-kubelet-dir\") pod \"aws-ebs-csi-driver-node-wtqvh\" (UID: \"21e8d664-4b0d-45cb-914f-7f55bb26d1f0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wtqvh"
Jul 16 11:03:31.192720 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.192281    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/21e8d664-4b0d-45cb-914f-7f55bb26d1f0-registration-dir\") pod \"aws-ebs-csi-driver-node-wtqvh\" (UID: \"21e8d664-4b0d-45cb-914f-7f55bb26d1f0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wtqvh"
Jul 16 11:03:31.192720 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.192274    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5baff3f6-85da-46fb-b42b-2e70e7d77cc4-host-var-lib-kubelet\") pod \"multus-zs9vs\" (UID: \"5baff3f6-85da-46fb-b42b-2e70e7d77cc4\") " pod="openshift-multus/multus-zs9vs"
Jul 16 11:03:31.192720 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.192320    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5baff3f6-85da-46fb-b42b-2e70e7d77cc4-host-var-lib-kubelet\") pod \"multus-zs9vs\" (UID: \"5baff3f6-85da-46fb-b42b-2e70e7d77cc4\") " pod="openshift-multus/multus-zs9vs"
Jul 16 11:03:31.192720 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.192328    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8de3fcaf-5b76-4d0f-8453-60b892832242-etc-systemd\") pod \"tuned-cr2x7\" (UID: \"8de3fcaf-5b76-4d0f-8453-60b892832242\") " pod="openshift-cluster-node-tuning-operator/tuned-cr2x7"
Jul 16 11:03:31.192720 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.192338    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/2d4103af-f06a-4592-83e0-8625e6267267-iptables-alerter-script\") pod \"iptables-alerter-x6lfk\" (UID: \"2d4103af-f06a-4592-83e0-8625e6267267\") " pod="openshift-network-operator/iptables-alerter-x6lfk"
Jul 16 11:03:31.192720 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.192353    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8de3fcaf-5b76-4d0f-8453-60b892832242-sys\") pod \"tuned-cr2x7\" (UID: \"8de3fcaf-5b76-4d0f-8453-60b892832242\") " pod="openshift-cluster-node-tuning-operator/tuned-cr2x7"
Jul 16 11:03:31.192720 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.192384    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qb2ts\" (UniqueName: \"kubernetes.io/projected/6fd1e802-78a8-4991-99ee-47cfea84ab62-kube-api-access-qb2ts\") pod \"multus-additional-cni-plugins-8r89z\" (UID: \"6fd1e802-78a8-4991-99ee-47cfea84ab62\") " pod="openshift-multus/multus-additional-cni-plugins-8r89z"
Jul 16 11:03:31.192720 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.192419    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h5wj2\" (UniqueName: \"kubernetes.io/projected/d4e7c597-9836-4a29-8591-e56938995710-kube-api-access-h5wj2\") pod \"network-metrics-daemon-qcl5d\" (UID: \"d4e7c597-9836-4a29-8591-e56938995710\") " pod="openshift-multus/network-metrics-daemon-qcl5d"
Jul 16 11:03:31.192720 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.192445    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0244f4b4-dd39-4870-97be-8378a99c93a2-host\") pod \"node-ca-6xfs6\" (UID: \"0244f4b4-dd39-4870-97be-8378a99c93a2\") " pod="openshift-image-registry/node-ca-6xfs6"
Jul 16 11:03:31.192720 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.192472    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6fd1e802-78a8-4991-99ee-47cfea84ab62-system-cni-dir\") pod \"multus-additional-cni-plugins-8r89z\" (UID: \"6fd1e802-78a8-4991-99ee-47cfea84ab62\") " pod="openshift-multus/multus-additional-cni-plugins-8r89z"
Jul 16 11:03:31.192720 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.192499    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/69aed169-e8af-429d-99c3-3b20be64e351-node-log\") pod \"ovnkube-node-2db6t\" (UID: \"69aed169-e8af-429d-99c3-3b20be64e351\") " pod="openshift-ovn-kubernetes/ovnkube-node-2db6t"
Jul 16 11:03:31.192720 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.192554    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/69aed169-e8af-429d-99c3-3b20be64e351-ovn-node-metrics-cert\") pod \"ovnkube-node-2db6t\" (UID: \"69aed169-e8af-429d-99c3-3b20be64e351\") " pod="openshift-ovn-kubernetes/ovnkube-node-2db6t"
Jul 16 11:03:31.192720 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.192618    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/69aed169-e8af-429d-99c3-3b20be64e351-ovnkube-script-lib\") pod \"ovnkube-node-2db6t\" (UID: \"69aed169-e8af-429d-99c3-3b20be64e351\") " pod="openshift-ovn-kubernetes/ovnkube-node-2db6t"
Jul 16 11:03:31.192720 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.192647    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6rq47\" (UniqueName: \"kubernetes.io/projected/69aed169-e8af-429d-99c3-3b20be64e351-kube-api-access-6rq47\") pod \"ovnkube-node-2db6t\" (UID: \"69aed169-e8af-429d-99c3-3b20be64e351\") " pod="openshift-ovn-kubernetes/ovnkube-node-2db6t"
Jul 16 11:03:31.192720 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.192673    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a3d44785-92d8-4a5e-bc2e-b91f4fa8b509-tmp-dir\") pod \"node-resolver-z5dtv\" (UID: \"a3d44785-92d8-4a5e-bc2e-b91f4fa8b509\") " pod="openshift-dns/node-resolver-z5dtv"
Jul 16 11:03:31.192720 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.192700    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/5baff3f6-85da-46fb-b42b-2e70e7d77cc4-host-run-k8s-cni-cncf-io\") pod \"multus-zs9vs\" (UID: \"5baff3f6-85da-46fb-b42b-2e70e7d77cc4\") " pod="openshift-multus/multus-zs9vs"
Jul 16 11:03:31.193478 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.192697    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/c24aaddd-b44c-4155-a7c5-fadb281f9b39-konnectivity-ca\") pod \"konnectivity-agent-qxpqt\" (UID: \"c24aaddd-b44c-4155-a7c5-fadb281f9b39\") " pod="kube-system/konnectivity-agent-qxpqt"
Jul 16 11:03:31.193478 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.192728    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5baff3f6-85da-46fb-b42b-2e70e7d77cc4-etc-kubernetes\") pod \"multus-zs9vs\" (UID: \"5baff3f6-85da-46fb-b42b-2e70e7d77cc4\") " pod="openshift-multus/multus-zs9vs"
Jul 16 11:03:31.193478 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.192752    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8de3fcaf-5b76-4d0f-8453-60b892832242-run\") pod \"tuned-cr2x7\" (UID: \"8de3fcaf-5b76-4d0f-8453-60b892832242\") " pod="openshift-cluster-node-tuning-operator/tuned-cr2x7"
Jul 16 11:03:31.193478 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.192777    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/69aed169-e8af-429d-99c3-3b20be64e351-host-kubelet\") pod \"ovnkube-node-2db6t\" (UID: \"69aed169-e8af-429d-99c3-3b20be64e351\") " pod="openshift-ovn-kubernetes/ovnkube-node-2db6t"
Jul 16 11:03:31.193478 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.192775    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/5baff3f6-85da-46fb-b42b-2e70e7d77cc4-host-run-k8s-cni-cncf-io\") pod \"multus-zs9vs\" (UID: \"5baff3f6-85da-46fb-b42b-2e70e7d77cc4\") " pod="openshift-multus/multus-zs9vs"
Jul 16 11:03:31.193478 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.192803    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/69aed169-e8af-429d-99c3-3b20be64e351-etc-openvswitch\") pod \"ovnkube-node-2db6t\" (UID: \"69aed169-e8af-429d-99c3-3b20be64e351\") " pod="openshift-ovn-kubernetes/ovnkube-node-2db6t"
Jul 16 11:03:31.193478 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.192797    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5baff3f6-85da-46fb-b42b-2e70e7d77cc4-etc-kubernetes\") pod \"multus-zs9vs\" (UID: \"5baff3f6-85da-46fb-b42b-2e70e7d77cc4\") " pod="openshift-multus/multus-zs9vs"
Jul 16 11:03:31.193478 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.192849    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/69aed169-e8af-429d-99c3-3b20be64e351-log-socket\") pod \"ovnkube-node-2db6t\" (UID: \"69aed169-e8af-429d-99c3-3b20be64e351\") " pod="openshift-ovn-kubernetes/ovnkube-node-2db6t"
Jul 16 11:03:31.193478 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.192889    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/69aed169-e8af-429d-99c3-3b20be64e351-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-2db6t\" (UID: \"69aed169-e8af-429d-99c3-3b20be64e351\") " pod="openshift-ovn-kubernetes/ovnkube-node-2db6t"
Jul 16 11:03:31.193478 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.192896    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5baff3f6-85da-46fb-b42b-2e70e7d77cc4-cni-binary-copy\") pod \"multus-zs9vs\" (UID: \"5baff3f6-85da-46fb-b42b-2e70e7d77cc4\") " pod="openshift-multus/multus-zs9vs"
Jul 16 11:03:31.193478 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.192921    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c88b6ce0-4926-4de9-942e-7f1b30c33e9e-dbus\") pod \"global-pull-secret-syncer-hlm2s\" (UID: \"c88b6ce0-4926-4de9-942e-7f1b30c33e9e\") " pod="kube-system/global-pull-secret-syncer-hlm2s"
Jul 16 11:03:31.193478 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.192951    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5baff3f6-85da-46fb-b42b-2e70e7d77cc4-multus-conf-dir\") pod \"multus-zs9vs\" (UID: \"5baff3f6-85da-46fb-b42b-2e70e7d77cc4\") " pod="openshift-multus/multus-zs9vs"
Jul 16 11:03:31.193478 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.193028    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8de3fcaf-5b76-4d0f-8453-60b892832242-var-lib-kubelet\") pod \"tuned-cr2x7\" (UID: \"8de3fcaf-5b76-4d0f-8453-60b892832242\") " pod="openshift-cluster-node-tuning-operator/tuned-cr2x7"
Jul 16 11:03:31.193478 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.193028    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5baff3f6-85da-46fb-b42b-2e70e7d77cc4-multus-conf-dir\") pod \"multus-zs9vs\" (UID: \"5baff3f6-85da-46fb-b42b-2e70e7d77cc4\") " pod="openshift-multus/multus-zs9vs"
Jul 16 11:03:31.193478 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.193087    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8de3fcaf-5b76-4d0f-8453-60b892832242-etc-tuned\") pod \"tuned-cr2x7\" (UID: \"8de3fcaf-5b76-4d0f-8453-60b892832242\") " pod="openshift-cluster-node-tuning-operator/tuned-cr2x7"
Jul 16 11:03:31.193478 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.193118    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/6fd1e802-78a8-4991-99ee-47cfea84ab62-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-8r89z\" (UID: \"6fd1e802-78a8-4991-99ee-47cfea84ab62\") " pod="openshift-multus/multus-additional-cni-plugins-8r89z"
Jul 16 11:03:31.193478 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.193155    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/69aed169-e8af-429d-99c3-3b20be64e351-var-lib-openvswitch\") pod \"ovnkube-node-2db6t\" (UID: \"69aed169-e8af-429d-99c3-3b20be64e351\") " pod="openshift-ovn-kubernetes/ovnkube-node-2db6t"
Jul 16 11:03:31.194362 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.193179    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/69aed169-e8af-429d-99c3-3b20be64e351-run-ovn\") pod \"ovnkube-node-2db6t\" (UID: \"69aed169-e8af-429d-99c3-3b20be64e351\") " pod="openshift-ovn-kubernetes/ovnkube-node-2db6t"
Jul 16 11:03:31.194362 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.193203    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/69aed169-e8af-429d-99c3-3b20be64e351-env-overrides\") pod \"ovnkube-node-2db6t\" (UID: \"69aed169-e8af-429d-99c3-3b20be64e351\") " pod="openshift-ovn-kubernetes/ovnkube-node-2db6t"
Jul 16 11:03:31.194362 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.193231    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zwq75\" (UniqueName: \"kubernetes.io/projected/a3d44785-92d8-4a5e-bc2e-b91f4fa8b509-kube-api-access-zwq75\") pod \"node-resolver-z5dtv\" (UID: \"a3d44785-92d8-4a5e-bc2e-b91f4fa8b509\") " pod="openshift-dns/node-resolver-z5dtv"
Jul 16 11:03:31.194362 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.193271    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/21e8d664-4b0d-45cb-914f-7f55bb26d1f0-socket-dir\") pod \"aws-ebs-csi-driver-node-wtqvh\" (UID: \"21e8d664-4b0d-45cb-914f-7f55bb26d1f0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wtqvh"
Jul 16 11:03:31.194362 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.193343    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/21e8d664-4b0d-45cb-914f-7f55bb26d1f0-etc-selinux\") pod \"aws-ebs-csi-driver-node-wtqvh\" (UID: \"21e8d664-4b0d-45cb-914f-7f55bb26d1f0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wtqvh"
Jul 16 11:03:31.194362 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.193383    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/21e8d664-4b0d-45cb-914f-7f55bb26d1f0-socket-dir\") pod \"aws-ebs-csi-driver-node-wtqvh\" (UID: \"21e8d664-4b0d-45cb-914f-7f55bb26d1f0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wtqvh"
Jul 16 11:03:31.194362 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.193386    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d4e7c597-9836-4a29-8591-e56938995710-metrics-certs\") pod \"network-metrics-daemon-qcl5d\" (UID: \"d4e7c597-9836-4a29-8591-e56938995710\") " pod="openshift-multus/network-metrics-daemon-qcl5d"
Jul 16 11:03:31.194362 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.193421    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8de3fcaf-5b76-4d0f-8453-60b892832242-etc-sysctl-d\") pod \"tuned-cr2x7\" (UID: \"8de3fcaf-5b76-4d0f-8453-60b892832242\") " pod="openshift-cluster-node-tuning-operator/tuned-cr2x7"
Jul 16 11:03:31.194362 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.193497    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/69aed169-e8af-429d-99c3-3b20be64e351-run-systemd\") pod \"ovnkube-node-2db6t\" (UID: \"69aed169-e8af-429d-99c3-3b20be64e351\") " pod="openshift-ovn-kubernetes/ovnkube-node-2db6t"
Jul 16 11:03:31.194362 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.193512    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/21e8d664-4b0d-45cb-914f-7f55bb26d1f0-etc-selinux\") pod \"aws-ebs-csi-driver-node-wtqvh\" (UID: \"21e8d664-4b0d-45cb-914f-7f55bb26d1f0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wtqvh"
Jul 16 11:03:31.194362 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:03:31.193529    2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 11:03:31.194362 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.193541    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/69aed169-e8af-429d-99c3-3b20be64e351-host-run-ovn-kubernetes\") pod \"ovnkube-node-2db6t\" (UID: \"69aed169-e8af-429d-99c3-3b20be64e351\") " pod="openshift-ovn-kubernetes/ovnkube-node-2db6t"
Jul 16 11:03:31.194362 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.193578    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5baff3f6-85da-46fb-b42b-2e70e7d77cc4-host-var-lib-cni-bin\") pod \"multus-zs9vs\" (UID: \"5baff3f6-85da-46fb-b42b-2e70e7d77cc4\") " pod="openshift-multus/multus-zs9vs"
Jul 16 11:03:31.194362 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.193636    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/69aed169-e8af-429d-99c3-3b20be64e351-host-cni-netd\") pod \"ovnkube-node-2db6t\" (UID: \"69aed169-e8af-429d-99c3-3b20be64e351\") " pod="openshift-ovn-kubernetes/ovnkube-node-2db6t"
Jul 16 11:03:31.194362 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.193654    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5baff3f6-85da-46fb-b42b-2e70e7d77cc4-host-var-lib-cni-bin\") pod \"multus-zs9vs\" (UID: \"5baff3f6-85da-46fb-b42b-2e70e7d77cc4\") " pod="openshift-multus/multus-zs9vs"
Jul 16 11:03:31.194362 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:03:31.193708    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d4e7c597-9836-4a29-8591-e56938995710-metrics-certs podName:d4e7c597-9836-4a29-8591-e56938995710 nodeName:}" failed. No retries permitted until 2026-07-16 11:03:31.693673404 +0000 UTC m=+4.327894874 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d4e7c597-9836-4a29-8591-e56938995710-metrics-certs") pod "network-metrics-daemon-qcl5d" (UID: "d4e7c597-9836-4a29-8591-e56938995710") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 11:03:31.195077 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.193732    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c88b6ce0-4926-4de9-942e-7f1b30c33e9e-original-pull-secret\") pod \"global-pull-secret-syncer-hlm2s\" (UID: \"c88b6ce0-4926-4de9-942e-7f1b30c33e9e\") " pod="kube-system/global-pull-secret-syncer-hlm2s"
Jul 16 11:03:31.195077 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.193771    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-85x99\" (UniqueName: \"kubernetes.io/projected/5baff3f6-85da-46fb-b42b-2e70e7d77cc4-kube-api-access-85x99\") pod \"multus-zs9vs\" (UID: \"5baff3f6-85da-46fb-b42b-2e70e7d77cc4\") " pod="openshift-multus/multus-zs9vs"
Jul 16 11:03:31.195077 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.193800    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2kcc2\" (UniqueName: \"kubernetes.io/projected/3d0ee8e8-4018-4963-b553-af9b6ecb0f80-kube-api-access-2kcc2\") pod \"network-check-target-rc7w4\" (UID: \"3d0ee8e8-4018-4963-b553-af9b6ecb0f80\") " pod="openshift-network-diagnostics/network-check-target-rc7w4"
Jul 16 11:03:31.195077 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.193825    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8de3fcaf-5b76-4d0f-8453-60b892832242-etc-kubernetes\") pod \"tuned-cr2x7\" (UID: \"8de3fcaf-5b76-4d0f-8453-60b892832242\") " pod="openshift-cluster-node-tuning-operator/tuned-cr2x7"
Jul 16 11:03:31.195077 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.193850    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/0244f4b4-dd39-4870-97be-8378a99c93a2-serviceca\") pod \"node-ca-6xfs6\" (UID: \"0244f4b4-dd39-4870-97be-8378a99c93a2\") " pod="openshift-image-registry/node-ca-6xfs6"
Jul 16 11:03:31.195077 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.193874    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6fd1e802-78a8-4991-99ee-47cfea84ab62-os-release\") pod \"multus-additional-cni-plugins-8r89z\" (UID: \"6fd1e802-78a8-4991-99ee-47cfea84ab62\") " pod="openshift-multus/multus-additional-cni-plugins-8r89z"
Jul 16 11:03:31.195077 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.193909    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/69aed169-e8af-429d-99c3-3b20be64e351-run-openvswitch\") pod \"ovnkube-node-2db6t\" (UID: \"69aed169-e8af-429d-99c3-3b20be64e351\") " pod="openshift-ovn-kubernetes/ovnkube-node-2db6t"
Jul 16 11:03:31.195077 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.193937    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/21e8d664-4b0d-45cb-914f-7f55bb26d1f0-device-dir\") pod \"aws-ebs-csi-driver-node-wtqvh\" (UID: \"21e8d664-4b0d-45cb-914f-7f55bb26d1f0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wtqvh"
Jul 16 11:03:31.195077 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.193999    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d8xds\" (UniqueName: \"kubernetes.io/projected/21e8d664-4b0d-45cb-914f-7f55bb26d1f0-kube-api-access-d8xds\") pod \"aws-ebs-csi-driver-node-wtqvh\" (UID: \"21e8d664-4b0d-45cb-914f-7f55bb26d1f0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wtqvh"
Jul 16 11:03:31.195077 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.194022    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/5baff3f6-85da-46fb-b42b-2e70e7d77cc4-multus-daemon-config\") pod \"multus-zs9vs\" (UID: \"5baff3f6-85da-46fb-b42b-2e70e7d77cc4\") " pod="openshift-multus/multus-zs9vs"
Jul 16 11:03:31.195077 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.194064    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/5baff3f6-85da-46fb-b42b-2e70e7d77cc4-host-run-multus-certs\") pod \"multus-zs9vs\" (UID: \"5baff3f6-85da-46fb-b42b-2e70e7d77cc4\") " pod="openshift-multus/multus-zs9vs"
Jul 16 11:03:31.195077 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.194105    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/21e8d664-4b0d-45cb-914f-7f55bb26d1f0-device-dir\") pod \"aws-ebs-csi-driver-node-wtqvh\" (UID: \"21e8d664-4b0d-45cb-914f-7f55bb26d1f0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wtqvh"
Jul 16 11:03:31.195077 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.194127    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/5baff3f6-85da-46fb-b42b-2e70e7d77cc4-host-run-multus-certs\") pod \"multus-zs9vs\" (UID: \"5baff3f6-85da-46fb-b42b-2e70e7d77cc4\") " pod="openshift-multus/multus-zs9vs"
Jul 16 11:03:31.195077 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.194318    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6fd1e802-78a8-4991-99ee-47cfea84ab62-cnibin\") pod \"multus-additional-cni-plugins-8r89z\" (UID: \"6fd1e802-78a8-4991-99ee-47cfea84ab62\") " pod="openshift-multus/multus-additional-cni-plugins-8r89z"
Jul 16 11:03:31.195077 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.194355    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6fd1e802-78a8-4991-99ee-47cfea84ab62-tuning-conf-dir\") pod \"multus-additional-cni-plugins-8r89z\" (UID: \"6fd1e802-78a8-4991-99ee-47cfea84ab62\") " pod="openshift-multus/multus-additional-cni-plugins-8r89z"
Jul 16 11:03:31.195077 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.194387    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/69aed169-e8af-429d-99c3-3b20be64e351-host-run-netns\") pod \"ovnkube-node-2db6t\" (UID: \"69aed169-e8af-429d-99c3-3b20be64e351\") " pod="openshift-ovn-kubernetes/ovnkube-node-2db6t"
Jul 16 11:03:31.195619 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.194429    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/69aed169-e8af-429d-99c3-3b20be64e351-ovnkube-config\") pod \"ovnkube-node-2db6t\" (UID: \"69aed169-e8af-429d-99c3-3b20be64e351\") " pod="openshift-ovn-kubernetes/ovnkube-node-2db6t"
Jul 16 11:03:31.195619 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.194468    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/21e8d664-4b0d-45cb-914f-7f55bb26d1f0-sys-fs\") pod \"aws-ebs-csi-driver-node-wtqvh\" (UID: \"21e8d664-4b0d-45cb-914f-7f55bb26d1f0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wtqvh"
Jul 16 11:03:31.195619 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.194486    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5baff3f6-85da-46fb-b42b-2e70e7d77cc4-host-run-netns\") pod \"multus-zs9vs\" (UID: \"5baff3f6-85da-46fb-b42b-2e70e7d77cc4\") " pod="openshift-multus/multus-zs9vs"
Jul 16 11:03:31.195619 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.194519    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vcz49\" (UniqueName: \"kubernetes.io/projected/0244f4b4-dd39-4870-97be-8378a99c93a2-kube-api-access-vcz49\") pod \"node-ca-6xfs6\" (UID: \"0244f4b4-dd39-4870-97be-8378a99c93a2\") " pod="openshift-image-registry/node-ca-6xfs6"
Jul 16 11:03:31.195619 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.194548    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/21e8d664-4b0d-45cb-914f-7f55bb26d1f0-sys-fs\") pod \"aws-ebs-csi-driver-node-wtqvh\" (UID: \"21e8d664-4b0d-45cb-914f-7f55bb26d1f0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wtqvh"
Jul 16 11:03:31.195619 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.194568    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/69aed169-e8af-429d-99c3-3b20be64e351-host-slash\") pod \"ovnkube-node-2db6t\" (UID: \"69aed169-e8af-429d-99c3-3b20be64e351\") " pod="openshift-ovn-kubernetes/ovnkube-node-2db6t"
Jul 16 11:03:31.195619 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.194591    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/5baff3f6-85da-46fb-b42b-2e70e7d77cc4-multus-daemon-config\") pod \"multus-zs9vs\" (UID: \"5baff3f6-85da-46fb-b42b-2e70e7d77cc4\") " pod="openshift-multus/multus-zs9vs"
Jul 16 11:03:31.195619 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.194603    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/69aed169-e8af-429d-99c3-3b20be64e351-host-cni-bin\") pod \"ovnkube-node-2db6t\" (UID: \"69aed169-e8af-429d-99c3-3b20be64e351\") " pod="openshift-ovn-kubernetes/ovnkube-node-2db6t"
Jul 16 11:03:31.195619 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.194618    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5baff3f6-85da-46fb-b42b-2e70e7d77cc4-host-run-netns\") pod \"multus-zs9vs\" (UID: \"5baff3f6-85da-46fb-b42b-2e70e7d77cc4\") " pod="openshift-multus/multus-zs9vs"
Jul 16 11:03:31.195619 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.194641    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c88b6ce0-4926-4de9-942e-7f1b30c33e9e-kubelet-config\") pod \"global-pull-secret-syncer-hlm2s\" (UID: \"c88b6ce0-4926-4de9-942e-7f1b30c33e9e\") " pod="kube-system/global-pull-secret-syncer-hlm2s"
Jul 16 11:03:31.195619 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.194662    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5baff3f6-85da-46fb-b42b-2e70e7d77cc4-multus-cni-dir\") pod \"multus-zs9vs\" (UID: \"5baff3f6-85da-46fb-b42b-2e70e7d77cc4\") " pod="openshift-multus/multus-zs9vs"
Jul 16 11:03:31.195619 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.194693    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8de3fcaf-5b76-4d0f-8453-60b892832242-host\") pod \"tuned-cr2x7\" (UID: \"8de3fcaf-5b76-4d0f-8453-60b892832242\") " pod="openshift-cluster-node-tuning-operator/tuned-cr2x7"
Jul 16 11:03:31.195619 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.194708    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/69aed169-e8af-429d-99c3-3b20be64e351-systemd-units\") pod \"ovnkube-node-2db6t\" (UID: \"69aed169-e8af-429d-99c3-3b20be64e351\") " pod="openshift-ovn-kubernetes/ovnkube-node-2db6t"
Jul 16 11:03:31.195619 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.194731    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8rbc4\" (UniqueName: \"kubernetes.io/projected/2d4103af-f06a-4592-83e0-8625e6267267-kube-api-access-8rbc4\") pod \"iptables-alerter-x6lfk\" (UID: \"2d4103af-f06a-4592-83e0-8625e6267267\") " pod="openshift-network-operator/iptables-alerter-x6lfk"
Jul 16 11:03:31.195619 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.194774    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8de3fcaf-5b76-4d0f-8453-60b892832242-tmp\") pod \"tuned-cr2x7\" (UID: \"8de3fcaf-5b76-4d0f-8453-60b892832242\") " pod="openshift-cluster-node-tuning-operator/tuned-cr2x7"
Jul 16 11:03:31.195619 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.194799    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7m82s\" (UniqueName: \"kubernetes.io/projected/8de3fcaf-5b76-4d0f-8453-60b892832242-kube-api-access-7m82s\") pod \"tuned-cr2x7\" (UID: \"8de3fcaf-5b76-4d0f-8453-60b892832242\") " pod="openshift-cluster-node-tuning-operator/tuned-cr2x7"
Jul 16 11:03:31.195619 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.194829    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5baff3f6-85da-46fb-b42b-2e70e7d77cc4-multus-cni-dir\") pod \"multus-zs9vs\" (UID: \"5baff3f6-85da-46fb-b42b-2e70e7d77cc4\") " pod="openshift-multus/multus-zs9vs"
Jul 16 11:03:31.197063 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.197036    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/c24aaddd-b44c-4155-a7c5-fadb281f9b39-agent-certs\") pod \"konnectivity-agent-qxpqt\" (UID: \"c24aaddd-b44c-4155-a7c5-fadb281f9b39\") " pod="kube-system/konnectivity-agent-qxpqt"
Jul 16 11:03:31.197527 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.197511    2583 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-m7gbm"
Jul 16 11:03:31.225558 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.225409    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-85x99\" (UniqueName: \"kubernetes.io/projected/5baff3f6-85da-46fb-b42b-2e70e7d77cc4-kube-api-access-85x99\") pod \"multus-zs9vs\" (UID: \"5baff3f6-85da-46fb-b42b-2e70e7d77cc4\") " pod="openshift-multus/multus-zs9vs"
Jul 16 11:03:31.225558 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.225413    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h5wj2\" (UniqueName: \"kubernetes.io/projected/d4e7c597-9836-4a29-8591-e56938995710-kube-api-access-h5wj2\") pod \"network-metrics-daemon-qcl5d\" (UID: \"d4e7c597-9836-4a29-8591-e56938995710\") " pod="openshift-multus/network-metrics-daemon-qcl5d"
Jul 16 11:03:31.227494 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.227465    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8rbc4\" (UniqueName: \"kubernetes.io/projected/2d4103af-f06a-4592-83e0-8625e6267267-kube-api-access-8rbc4\") pod \"iptables-alerter-x6lfk\" (UID: \"2d4103af-f06a-4592-83e0-8625e6267267\") " pod="openshift-network-operator/iptables-alerter-x6lfk"
Jul 16 11:03:31.227636 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.227518    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d8xds\" (UniqueName: \"kubernetes.io/projected/21e8d664-4b0d-45cb-914f-7f55bb26d1f0-kube-api-access-d8xds\") pod \"aws-ebs-csi-driver-node-wtqvh\" (UID: \"21e8d664-4b0d-45cb-914f-7f55bb26d1f0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wtqvh"
Jul 16 11:03:31.229989 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.229935    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-28.ec2.internal" event={"ID":"b31dba48238fdc286bfab6bd4db2f24f","Type":"ContainerStarted","Data":"ddb629d3e4f03afa0cb14e1f89f409e76c74fc8a8a322c79ef771feb2ec2dc72"}
Jul 16 11:03:31.295420 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.295370    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/69aed169-e8af-429d-99c3-3b20be64e351-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-2db6t\" (UID: \"69aed169-e8af-429d-99c3-3b20be64e351\") " pod="openshift-ovn-kubernetes/ovnkube-node-2db6t"
Jul 16 11:03:31.295420 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.295413    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c88b6ce0-4926-4de9-942e-7f1b30c33e9e-dbus\") pod \"global-pull-secret-syncer-hlm2s\" (UID: \"c88b6ce0-4926-4de9-942e-7f1b30c33e9e\") " pod="kube-system/global-pull-secret-syncer-hlm2s"
Jul 16 11:03:31.295696 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.295434    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8de3fcaf-5b76-4d0f-8453-60b892832242-var-lib-kubelet\") pod \"tuned-cr2x7\" (UID: \"8de3fcaf-5b76-4d0f-8453-60b892832242\") " pod="openshift-cluster-node-tuning-operator/tuned-cr2x7"
Jul 16 11:03:31.295696 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.295534    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c88b6ce0-4926-4de9-942e-7f1b30c33e9e-dbus\") pod \"global-pull-secret-syncer-hlm2s\" (UID: \"c88b6ce0-4926-4de9-942e-7f1b30c33e9e\") " pod="kube-system/global-pull-secret-syncer-hlm2s"
Jul 16 11:03:31.295696 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.295516    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/69aed169-e8af-429d-99c3-3b20be64e351-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-2db6t\" (UID: \"69aed169-e8af-429d-99c3-3b20be64e351\") " pod="openshift-ovn-kubernetes/ovnkube-node-2db6t"
Jul 16 11:03:31.295696 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.295585    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8de3fcaf-5b76-4d0f-8453-60b892832242-var-lib-kubelet\") pod \"tuned-cr2x7\" (UID: \"8de3fcaf-5b76-4d0f-8453-60b892832242\") " pod="openshift-cluster-node-tuning-operator/tuned-cr2x7"
Jul 16 11:03:31.295696 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.295638    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8de3fcaf-5b76-4d0f-8453-60b892832242-etc-tuned\") pod \"tuned-cr2x7\" (UID: \"8de3fcaf-5b76-4d0f-8453-60b892832242\") " pod="openshift-cluster-node-tuning-operator/tuned-cr2x7"
Jul 16 11:03:31.295696 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.295666    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/6fd1e802-78a8-4991-99ee-47cfea84ab62-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-8r89z\" (UID: \"6fd1e802-78a8-4991-99ee-47cfea84ab62\") " pod="openshift-multus/multus-additional-cni-plugins-8r89z"
Jul 16 11:03:31.295696 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.295687    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/69aed169-e8af-429d-99c3-3b20be64e351-var-lib-openvswitch\") pod \"ovnkube-node-2db6t\" (UID: \"69aed169-e8af-429d-99c3-3b20be64e351\") " pod="openshift-ovn-kubernetes/ovnkube-node-2db6t"
Jul 16 11:03:31.296024 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.295710    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/69aed169-e8af-429d-99c3-3b20be64e351-run-ovn\") pod \"ovnkube-node-2db6t\" (UID: \"69aed169-e8af-429d-99c3-3b20be64e351\") " pod="openshift-ovn-kubernetes/ovnkube-node-2db6t"
Jul 16 11:03:31.296024 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.295733    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/69aed169-e8af-429d-99c3-3b20be64e351-env-overrides\") pod \"ovnkube-node-2db6t\" (UID: \"69aed169-e8af-429d-99c3-3b20be64e351\") " pod="openshift-ovn-kubernetes/ovnkube-node-2db6t"
Jul 16 11:03:31.296024 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.295760    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zwq75\" (UniqueName: \"kubernetes.io/projected/a3d44785-92d8-4a5e-bc2e-b91f4fa8b509-kube-api-access-zwq75\") pod \"node-resolver-z5dtv\" (UID: \"a3d44785-92d8-4a5e-bc2e-b91f4fa8b509\") " pod="openshift-dns/node-resolver-z5dtv"
Jul 16 11:03:31.296024 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.295768    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/69aed169-e8af-429d-99c3-3b20be64e351-var-lib-openvswitch\") pod \"ovnkube-node-2db6t\" (UID: \"69aed169-e8af-429d-99c3-3b20be64e351\") " pod="openshift-ovn-kubernetes/ovnkube-node-2db6t"
Jul 16 11:03:31.296024 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.295820    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8de3fcaf-5b76-4d0f-8453-60b892832242-etc-sysctl-d\") pod \"tuned-cr2x7\" (UID: \"8de3fcaf-5b76-4d0f-8453-60b892832242\") " pod="openshift-cluster-node-tuning-operator/tuned-cr2x7"
Jul 16 11:03:31.296024 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.295846    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/69aed169-e8af-429d-99c3-3b20be64e351-run-systemd\") pod \"ovnkube-node-2db6t\" (UID: \"69aed169-e8af-429d-99c3-3b20be64e351\") " pod="openshift-ovn-kubernetes/ovnkube-node-2db6t"
Jul 16 11:03:31.296024 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.295852    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/69aed169-e8af-429d-99c3-3b20be64e351-run-ovn\") pod \"ovnkube-node-2db6t\" (UID: \"69aed169-e8af-429d-99c3-3b20be64e351\") " pod="openshift-ovn-kubernetes/ovnkube-node-2db6t"
Jul 16 11:03:31.296024 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.295872    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/69aed169-e8af-429d-99c3-3b20be64e351-host-run-ovn-kubernetes\") pod \"ovnkube-node-2db6t\" (UID: \"69aed169-e8af-429d-99c3-3b20be64e351\") " pod="openshift-ovn-kubernetes/ovnkube-node-2db6t"
Jul 16 11:03:31.296024 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.295900    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/69aed169-e8af-429d-99c3-3b20be64e351-host-cni-netd\") pod \"ovnkube-node-2db6t\" (UID: \"69aed169-e8af-429d-99c3-3b20be64e351\") " pod="openshift-ovn-kubernetes/ovnkube-node-2db6t"
Jul 16 11:03:31.296024 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.295927    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c88b6ce0-4926-4de9-942e-7f1b30c33e9e-original-pull-secret\") pod \"global-pull-secret-syncer-hlm2s\" (UID: \"c88b6ce0-4926-4de9-942e-7f1b30c33e9e\") " pod="kube-system/global-pull-secret-syncer-hlm2s"
Jul 16 11:03:31.296024 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.295957    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2kcc2\" (UniqueName: \"kubernetes.io/projected/3d0ee8e8-4018-4963-b553-af9b6ecb0f80-kube-api-access-2kcc2\") pod \"network-check-target-rc7w4\" (UID: \"3d0ee8e8-4018-4963-b553-af9b6ecb0f80\") " pod="openshift-network-diagnostics/network-check-target-rc7w4"
Jul 16 11:03:31.296024 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.295983    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8de3fcaf-5b76-4d0f-8453-60b892832242-etc-kubernetes\") pod \"tuned-cr2x7\" (UID: \"8de3fcaf-5b76-4d0f-8453-60b892832242\") " pod="openshift-cluster-node-tuning-operator/tuned-cr2x7"
Jul 16 11:03:31.296024 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.296007    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/0244f4b4-dd39-4870-97be-8378a99c93a2-serviceca\") pod \"node-ca-6xfs6\" (UID: \"0244f4b4-dd39-4870-97be-8378a99c93a2\") " pod="openshift-image-registry/node-ca-6xfs6"
Jul 16 11:03:31.296024 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.296016    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8de3fcaf-5b76-4d0f-8453-60b892832242-etc-sysctl-d\") pod \"tuned-cr2x7\" (UID: \"8de3fcaf-5b76-4d0f-8453-60b892832242\") " pod="openshift-cluster-node-tuning-operator/tuned-cr2x7"
Jul 16 11:03:31.296024 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.296033    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6fd1e802-78a8-4991-99ee-47cfea84ab62-os-release\") pod \"multus-additional-cni-plugins-8r89z\" (UID: \"6fd1e802-78a8-4991-99ee-47cfea84ab62\") " pod="openshift-multus/multus-additional-cni-plugins-8r89z"
Jul 16 11:03:31.296803 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.296059    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/69aed169-e8af-429d-99c3-3b20be64e351-run-openvswitch\") pod \"ovnkube-node-2db6t\" (UID: \"69aed169-e8af-429d-99c3-3b20be64e351\") " pod="openshift-ovn-kubernetes/ovnkube-node-2db6t"
Jul 16 11:03:31.296803 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.296089    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6fd1e802-78a8-4991-99ee-47cfea84ab62-cnibin\") pod \"multus-additional-cni-plugins-8r89z\" (UID: \"6fd1e802-78a8-4991-99ee-47cfea84ab62\") " pod="openshift-multus/multus-additional-cni-plugins-8r89z"
Jul 16 11:03:31.296803 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:03:31.296101    2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 16 11:03:31.296803 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.296113    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6fd1e802-78a8-4991-99ee-47cfea84ab62-tuning-conf-dir\") pod \"multus-additional-cni-plugins-8r89z\" (UID: \"6fd1e802-78a8-4991-99ee-47cfea84ab62\") " pod="openshift-multus/multus-additional-cni-plugins-8r89z"
Jul 16 11:03:31.296803 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.296116    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8de3fcaf-5b76-4d0f-8453-60b892832242-etc-kubernetes\") pod \"tuned-cr2x7\" (UID: \"8de3fcaf-5b76-4d0f-8453-60b892832242\") " pod="openshift-cluster-node-tuning-operator/tuned-cr2x7"
Jul 16 11:03:31.296803 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.296142    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/69aed169-e8af-429d-99c3-3b20be64e351-host-run-netns\") pod \"ovnkube-node-2db6t\" (UID: \"69aed169-e8af-429d-99c3-3b20be64e351\") " pod="openshift-ovn-kubernetes/ovnkube-node-2db6t"
Jul 16 11:03:31.296803 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:03:31.296168    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c88b6ce0-4926-4de9-942e-7f1b30c33e9e-original-pull-secret podName:c88b6ce0-4926-4de9-942e-7f1b30c33e9e nodeName:}" failed. No retries permitted until 2026-07-16 11:03:31.796143752 +0000 UTC m=+4.430365203 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c88b6ce0-4926-4de9-942e-7f1b30c33e9e-original-pull-secret") pod "global-pull-secret-syncer-hlm2s" (UID: "c88b6ce0-4926-4de9-942e-7f1b30c33e9e") : object "kube-system"/"original-pull-secret" not registered
Jul 16 11:03:31.296803 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.296205    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/69aed169-e8af-429d-99c3-3b20be64e351-run-systemd\") pod \"ovnkube-node-2db6t\" (UID: \"69aed169-e8af-429d-99c3-3b20be64e351\") " pod="openshift-ovn-kubernetes/ovnkube-node-2db6t"
Jul 16 11:03:31.296803 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.296245    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/69aed169-e8af-429d-99c3-3b20be64e351-host-run-ovn-kubernetes\") pod \"ovnkube-node-2db6t\" (UID: \"69aed169-e8af-429d-99c3-3b20be64e351\") " pod="openshift-ovn-kubernetes/ovnkube-node-2db6t"
Jul 16 11:03:31.296803 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.296286    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/69aed169-e8af-429d-99c3-3b20be64e351-host-cni-netd\") pod \"ovnkube-node-2db6t\" (UID: \"69aed169-e8af-429d-99c3-3b20be64e351\") " pod="openshift-ovn-kubernetes/ovnkube-node-2db6t"
Jul 16 11:03:31.296803 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.296283    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6fd1e802-78a8-4991-99ee-47cfea84ab62-cnibin\") pod \"multus-additional-cni-plugins-8r89z\" (UID: \"6fd1e802-78a8-4991-99ee-47cfea84ab62\") " pod="openshift-multus/multus-additional-cni-plugins-8r89z"
Jul 16 11:03:31.296803 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.296341    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/69aed169-e8af-429d-99c3-3b20be64e351-ovnkube-config\") pod \"ovnkube-node-2db6t\" (UID: \"69aed169-e8af-429d-99c3-3b20be64e351\") " pod="openshift-ovn-kubernetes/ovnkube-node-2db6t"
Jul 16 11:03:31.296803 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.296370    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vcz49\" (UniqueName: \"kubernetes.io/projected/0244f4b4-dd39-4870-97be-8378a99c93a2-kube-api-access-vcz49\") pod \"node-ca-6xfs6\" (UID: \"0244f4b4-dd39-4870-97be-8378a99c93a2\") " pod="openshift-image-registry/node-ca-6xfs6"
Jul 16 11:03:31.296803 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.296372    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6fd1e802-78a8-4991-99ee-47cfea84ab62-os-release\") pod \"multus-additional-cni-plugins-8r89z\" (UID: \"6fd1e802-78a8-4991-99ee-47cfea84ab62\") " pod="openshift-multus/multus-additional-cni-plugins-8r89z"
Jul 16 11:03:31.296803 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.296378    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/6fd1e802-78a8-4991-99ee-47cfea84ab62-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-8r89z\" (UID: \"6fd1e802-78a8-4991-99ee-47cfea84ab62\") " pod="openshift-multus/multus-additional-cni-plugins-8r89z"
Jul 16 11:03:31.296803 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.296395    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/69aed169-e8af-429d-99c3-3b20be64e351-host-slash\") pod \"ovnkube-node-2db6t\" (UID: \"69aed169-e8af-429d-99c3-3b20be64e351\") " pod="openshift-ovn-kubernetes/ovnkube-node-2db6t"
Jul 16 11:03:31.296803 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.296422    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/69aed169-e8af-429d-99c3-3b20be64e351-host-cni-bin\") pod \"ovnkube-node-2db6t\" (UID: \"69aed169-e8af-429d-99c3-3b20be64e351\") " pod="openshift-ovn-kubernetes/ovnkube-node-2db6t"
Jul 16 11:03:31.297645 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.296442    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/69aed169-e8af-429d-99c3-3b20be64e351-run-openvswitch\") pod \"ovnkube-node-2db6t\" (UID: \"69aed169-e8af-429d-99c3-3b20be64e351\") " pod="openshift-ovn-kubernetes/ovnkube-node-2db6t"
Jul 16 11:03:31.297645 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.296391    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/69aed169-e8af-429d-99c3-3b20be64e351-env-overrides\") pod \"ovnkube-node-2db6t\" (UID: \"69aed169-e8af-429d-99c3-3b20be64e351\") " pod="openshift-ovn-kubernetes/ovnkube-node-2db6t"
Jul 16 11:03:31.297645 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.296464    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/69aed169-e8af-429d-99c3-3b20be64e351-host-cni-bin\") pod \"ovnkube-node-2db6t\" (UID: \"69aed169-e8af-429d-99c3-3b20be64e351\") " pod="openshift-ovn-kubernetes/ovnkube-node-2db6t"
Jul 16 11:03:31.297645 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.296495    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/69aed169-e8af-429d-99c3-3b20be64e351-host-slash\") pod \"ovnkube-node-2db6t\" (UID: \"69aed169-e8af-429d-99c3-3b20be64e351\") " pod="openshift-ovn-kubernetes/ovnkube-node-2db6t"
Jul 16 11:03:31.297645 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.296512    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6fd1e802-78a8-4991-99ee-47cfea84ab62-tuning-conf-dir\") pod \"multus-additional-cni-plugins-8r89z\" (UID: \"6fd1e802-78a8-4991-99ee-47cfea84ab62\") " pod="openshift-multus/multus-additional-cni-plugins-8r89z"
Jul 16 11:03:31.297645 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.296514    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/69aed169-e8af-429d-99c3-3b20be64e351-host-run-netns\") pod \"ovnkube-node-2db6t\" (UID: \"69aed169-e8af-429d-99c3-3b20be64e351\") " pod="openshift-ovn-kubernetes/ovnkube-node-2db6t"
Jul 16 11:03:31.297645 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.296563    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/0244f4b4-dd39-4870-97be-8378a99c93a2-serviceca\") pod \"node-ca-6xfs6\" (UID: \"0244f4b4-dd39-4870-97be-8378a99c93a2\") " pod="openshift-image-registry/node-ca-6xfs6"
Jul 16 11:03:31.297645 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.296564    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c88b6ce0-4926-4de9-942e-7f1b30c33e9e-kubelet-config\") pod \"global-pull-secret-syncer-hlm2s\" (UID: \"c88b6ce0-4926-4de9-942e-7f1b30c33e9e\") " pod="kube-system/global-pull-secret-syncer-hlm2s"
Jul 16 11:03:31.297645 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.296605    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c88b6ce0-4926-4de9-942e-7f1b30c33e9e-kubelet-config\") pod \"global-pull-secret-syncer-hlm2s\" (UID: \"c88b6ce0-4926-4de9-942e-7f1b30c33e9e\") " pod="kube-system/global-pull-secret-syncer-hlm2s"
Jul 16 11:03:31.297645 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.296609    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8de3fcaf-5b76-4d0f-8453-60b892832242-host\") pod \"tuned-cr2x7\" (UID: \"8de3fcaf-5b76-4d0f-8453-60b892832242\") " pod="openshift-cluster-node-tuning-operator/tuned-cr2x7"
Jul 16 11:03:31.297645 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.296646    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8de3fcaf-5b76-4d0f-8453-60b892832242-host\") pod \"tuned-cr2x7\" (UID: \"8de3fcaf-5b76-4d0f-8453-60b892832242\") " pod="openshift-cluster-node-tuning-operator/tuned-cr2x7"
Jul 16 11:03:31.297645 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.296661    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/69aed169-e8af-429d-99c3-3b20be64e351-systemd-units\") pod \"ovnkube-node-2db6t\" (UID: \"69aed169-e8af-429d-99c3-3b20be64e351\") " pod="openshift-ovn-kubernetes/ovnkube-node-2db6t"
Jul 16 11:03:31.297645 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.296712    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8de3fcaf-5b76-4d0f-8453-60b892832242-tmp\") pod \"tuned-cr2x7\" (UID: \"8de3fcaf-5b76-4d0f-8453-60b892832242\") " pod="openshift-cluster-node-tuning-operator/tuned-cr2x7"
Jul 16 11:03:31.297645 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.296718    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/69aed169-e8af-429d-99c3-3b20be64e351-systemd-units\") pod \"ovnkube-node-2db6t\" (UID: \"69aed169-e8af-429d-99c3-3b20be64e351\") " pod="openshift-ovn-kubernetes/ovnkube-node-2db6t"
Jul 16 11:03:31.297645 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.296740    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7m82s\" (UniqueName: \"kubernetes.io/projected/8de3fcaf-5b76-4d0f-8453-60b892832242-kube-api-access-7m82s\") pod \"tuned-cr2x7\" (UID: \"8de3fcaf-5b76-4d0f-8453-60b892832242\") " pod="openshift-cluster-node-tuning-operator/tuned-cr2x7"
Jul 16 11:03:31.297645 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.296849    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/a3d44785-92d8-4a5e-bc2e-b91f4fa8b509-hosts-file\") pod \"node-resolver-z5dtv\" (UID: \"a3d44785-92d8-4a5e-bc2e-b91f4fa8b509\") " pod="openshift-dns/node-resolver-z5dtv"
Jul 16 11:03:31.297645 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.296879    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8de3fcaf-5b76-4d0f-8453-60b892832242-etc-sysctl-conf\") pod \"tuned-cr2x7\" (UID: \"8de3fcaf-5b76-4d0f-8453-60b892832242\") " pod="openshift-cluster-node-tuning-operator/tuned-cr2x7"
Jul 16 11:03:31.297645 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.296907    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/69aed169-e8af-429d-99c3-3b20be64e351-ovnkube-config\") pod \"ovnkube-node-2db6t\" (UID: \"69aed169-e8af-429d-99c3-3b20be64e351\") " pod="openshift-ovn-kubernetes/ovnkube-node-2db6t"
Jul 16 11:03:31.298562 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.296914    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8de3fcaf-5b76-4d0f-8453-60b892832242-etc-sysconfig\") pod \"tuned-cr2x7\" (UID: \"8de3fcaf-5b76-4d0f-8453-60b892832242\") " pod="openshift-cluster-node-tuning-operator/tuned-cr2x7"
Jul 16 11:03:31.298562 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.296965    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8de3fcaf-5b76-4d0f-8453-60b892832242-etc-sysconfig\") pod \"tuned-cr2x7\" (UID: \"8de3fcaf-5b76-4d0f-8453-60b892832242\") " pod="openshift-cluster-node-tuning-operator/tuned-cr2x7"
Jul 16 11:03:31.298562 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.296976    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8de3fcaf-5b76-4d0f-8453-60b892832242-etc-modprobe-d\") pod \"tuned-cr2x7\" (UID: \"8de3fcaf-5b76-4d0f-8453-60b892832242\") " pod="openshift-cluster-node-tuning-operator/tuned-cr2x7"
Jul 16 11:03:31.298562 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.297005    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8de3fcaf-5b76-4d0f-8453-60b892832242-lib-modules\") pod \"tuned-cr2x7\" (UID: \"8de3fcaf-5b76-4d0f-8453-60b892832242\") " pod="openshift-cluster-node-tuning-operator/tuned-cr2x7"
Jul 16 11:03:31.298562 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.297016    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/a3d44785-92d8-4a5e-bc2e-b91f4fa8b509-hosts-file\") pod \"node-resolver-z5dtv\" (UID: \"a3d44785-92d8-4a5e-bc2e-b91f4fa8b509\") " pod="openshift-dns/node-resolver-z5dtv"
Jul 16 11:03:31.298562 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.297033    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/6fd1e802-78a8-4991-99ee-47cfea84ab62-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-8r89z\" (UID: \"6fd1e802-78a8-4991-99ee-47cfea84ab62\") " pod="openshift-multus/multus-additional-cni-plugins-8r89z"
Jul 16 11:03:31.298562 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.297066    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6fd1e802-78a8-4991-99ee-47cfea84ab62-cni-binary-copy\") pod \"multus-additional-cni-plugins-8r89z\" (UID: \"6fd1e802-78a8-4991-99ee-47cfea84ab62\") " pod="openshift-multus/multus-additional-cni-plugins-8r89z"
Jul 16 11:03:31.298562 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.297101    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8de3fcaf-5b76-4d0f-8453-60b892832242-etc-systemd\") pod \"tuned-cr2x7\" (UID: \"8de3fcaf-5b76-4d0f-8453-60b892832242\") " pod="openshift-cluster-node-tuning-operator/tuned-cr2x7"
Jul 16 11:03:31.298562 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.297124    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8de3fcaf-5b76-4d0f-8453-60b892832242-sys\") pod \"tuned-cr2x7\" (UID: \"8de3fcaf-5b76-4d0f-8453-60b892832242\") " pod="openshift-cluster-node-tuning-operator/tuned-cr2x7"
Jul 16 11:03:31.298562 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.297136    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8de3fcaf-5b76-4d0f-8453-60b892832242-etc-modprobe-d\") pod \"tuned-cr2x7\" (UID: \"8de3fcaf-5b76-4d0f-8453-60b892832242\") " pod="openshift-cluster-node-tuning-operator/tuned-cr2x7"
Jul 16 11:03:31.298562 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.297150    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qb2ts\" (UniqueName: \"kubernetes.io/projected/6fd1e802-78a8-4991-99ee-47cfea84ab62-kube-api-access-qb2ts\") pod \"multus-additional-cni-plugins-8r89z\" (UID: \"6fd1e802-78a8-4991-99ee-47cfea84ab62\") " pod="openshift-multus/multus-additional-cni-plugins-8r89z"
Jul 16 11:03:31.298562 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.297154    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8de3fcaf-5b76-4d0f-8453-60b892832242-etc-sysctl-conf\") pod \"tuned-cr2x7\" (UID: \"8de3fcaf-5b76-4d0f-8453-60b892832242\") " pod="openshift-cluster-node-tuning-operator/tuned-cr2x7"
Jul 16 11:03:31.298562 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.297178    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0244f4b4-dd39-4870-97be-8378a99c93a2-host\") pod \"node-ca-6xfs6\" (UID: \"0244f4b4-dd39-4870-97be-8378a99c93a2\") " pod="openshift-image-registry/node-ca-6xfs6"
Jul 16 11:03:31.298562 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.297197    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8de3fcaf-5b76-4d0f-8453-60b892832242-lib-modules\") pod \"tuned-cr2x7\" (UID: \"8de3fcaf-5b76-4d0f-8453-60b892832242\") " pod="openshift-cluster-node-tuning-operator/tuned-cr2x7"
Jul 16 11:03:31.298562 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.297206    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6fd1e802-78a8-4991-99ee-47cfea84ab62-system-cni-dir\") pod \"multus-additional-cni-plugins-8r89z\" (UID: \"6fd1e802-78a8-4991-99ee-47cfea84ab62\") " pod="openshift-multus/multus-additional-cni-plugins-8r89z"
Jul 16 11:03:31.298562 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.297243    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/69aed169-e8af-429d-99c3-3b20be64e351-node-log\") pod \"ovnkube-node-2db6t\" (UID: \"69aed169-e8af-429d-99c3-3b20be64e351\") " pod="openshift-ovn-kubernetes/ovnkube-node-2db6t"
Jul 16 11:03:31.298562 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.297255    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6fd1e802-78a8-4991-99ee-47cfea84ab62-system-cni-dir\") pod \"multus-additional-cni-plugins-8r89z\" (UID: \"6fd1e802-78a8-4991-99ee-47cfea84ab62\") " pod="openshift-multus/multus-additional-cni-plugins-8r89z"
Jul 16 11:03:31.299494 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.297269    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/69aed169-e8af-429d-99c3-3b20be64e351-ovn-node-metrics-cert\") pod \"ovnkube-node-2db6t\" (UID: \"69aed169-e8af-429d-99c3-3b20be64e351\") " pod="openshift-ovn-kubernetes/ovnkube-node-2db6t"
Jul 16 11:03:31.299494 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.297297    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/69aed169-e8af-429d-99c3-3b20be64e351-ovnkube-script-lib\") pod \"ovnkube-node-2db6t\" (UID: \"69aed169-e8af-429d-99c3-3b20be64e351\") " pod="openshift-ovn-kubernetes/ovnkube-node-2db6t"
Jul 16 11:03:31.299494 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.297326    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6rq47\" (UniqueName: \"kubernetes.io/projected/69aed169-e8af-429d-99c3-3b20be64e351-kube-api-access-6rq47\") pod \"ovnkube-node-2db6t\" (UID: \"69aed169-e8af-429d-99c3-3b20be64e351\") " pod="openshift-ovn-kubernetes/ovnkube-node-2db6t"
Jul 16 11:03:31.299494 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.297346    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8de3fcaf-5b76-4d0f-8453-60b892832242-etc-systemd\") pod \"tuned-cr2x7\" (UID: \"8de3fcaf-5b76-4d0f-8453-60b892832242\") " pod="openshift-cluster-node-tuning-operator/tuned-cr2x7"
Jul 16 11:03:31.299494 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.297350    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a3d44785-92d8-4a5e-bc2e-b91f4fa8b509-tmp-dir\") pod \"node-resolver-z5dtv\" (UID: \"a3d44785-92d8-4a5e-bc2e-b91f4fa8b509\") " pod="openshift-dns/node-resolver-z5dtv"
Jul 16 11:03:31.299494 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.297386    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8de3fcaf-5b76-4d0f-8453-60b892832242-run\") pod \"tuned-cr2x7\" (UID: \"8de3fcaf-5b76-4d0f-8453-60b892832242\") " pod="openshift-cluster-node-tuning-operator/tuned-cr2x7"
Jul 16 11:03:31.299494 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.297414    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/69aed169-e8af-429d-99c3-3b20be64e351-host-kubelet\") pod \"ovnkube-node-2db6t\" (UID: \"69aed169-e8af-429d-99c3-3b20be64e351\") " pod="openshift-ovn-kubernetes/ovnkube-node-2db6t"
Jul 16 11:03:31.299494 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.297460    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/69aed169-e8af-429d-99c3-3b20be64e351-etc-openvswitch\") pod \"ovnkube-node-2db6t\" (UID: \"69aed169-e8af-429d-99c3-3b20be64e351\") " pod="openshift-ovn-kubernetes/ovnkube-node-2db6t"
Jul 16 11:03:31.299494 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.297487    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/69aed169-e8af-429d-99c3-3b20be64e351-log-socket\") pod \"ovnkube-node-2db6t\" (UID: \"69aed169-e8af-429d-99c3-3b20be64e351\") " pod="openshift-ovn-kubernetes/ovnkube-node-2db6t"
Jul 16 11:03:31.299494 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.297565    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/69aed169-e8af-429d-99c3-3b20be64e351-log-socket\") pod \"ovnkube-node-2db6t\" (UID: \"69aed169-e8af-429d-99c3-3b20be64e351\") " pod="openshift-ovn-kubernetes/ovnkube-node-2db6t"
Jul 16 11:03:31.299494 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.297613    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8de3fcaf-5b76-4d0f-8453-60b892832242-sys\") pod \"tuned-cr2x7\" (UID: \"8de3fcaf-5b76-4d0f-8453-60b892832242\") " pod="openshift-cluster-node-tuning-operator/tuned-cr2x7"
Jul 16 11:03:31.299494 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.297681    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a3d44785-92d8-4a5e-bc2e-b91f4fa8b509-tmp-dir\") pod \"node-resolver-z5dtv\" (UID: \"a3d44785-92d8-4a5e-bc2e-b91f4fa8b509\") " pod="openshift-dns/node-resolver-z5dtv"
Jul 16 11:03:31.299494 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.297723    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6fd1e802-78a8-4991-99ee-47cfea84ab62-cni-binary-copy\") pod \"multus-additional-cni-plugins-8r89z\" (UID: \"6fd1e802-78a8-4991-99ee-47cfea84ab62\") " pod="openshift-multus/multus-additional-cni-plugins-8r89z"
Jul 16 11:03:31.299494 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.297775    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/6fd1e802-78a8-4991-99ee-47cfea84ab62-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-8r89z\" (UID: \"6fd1e802-78a8-4991-99ee-47cfea84ab62\") " pod="openshift-multus/multus-additional-cni-plugins-8r89z"
Jul 16 11:03:31.299494 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.297980    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/69aed169-e8af-429d-99c3-3b20be64e351-node-log\") pod \"ovnkube-node-2db6t\" (UID: \"69aed169-e8af-429d-99c3-3b20be64e351\") " pod="openshift-ovn-kubernetes/ovnkube-node-2db6t"
Jul 16 11:03:31.299494 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.298039    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8de3fcaf-5b76-4d0f-8453-60b892832242-run\") pod \"tuned-cr2x7\" (UID: \"8de3fcaf-5b76-4d0f-8453-60b892832242\") " pod="openshift-cluster-node-tuning-operator/tuned-cr2x7"
Jul 16 11:03:31.299494 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.298052    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/69aed169-e8af-429d-99c3-3b20be64e351-host-kubelet\") pod \"ovnkube-node-2db6t\" (UID: \"69aed169-e8af-429d-99c3-3b20be64e351\") " pod="openshift-ovn-kubernetes/ovnkube-node-2db6t"
Jul 16 11:03:31.299494 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.298082    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0244f4b4-dd39-4870-97be-8378a99c93a2-host\") pod \"node-ca-6xfs6\" (UID: \"0244f4b4-dd39-4870-97be-8378a99c93a2\") " pod="openshift-image-registry/node-ca-6xfs6"
Jul 16 11:03:31.300469 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.298106    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/69aed169-e8af-429d-99c3-3b20be64e351-etc-openvswitch\") pod \"ovnkube-node-2db6t\" (UID: \"69aed169-e8af-429d-99c3-3b20be64e351\") " pod="openshift-ovn-kubernetes/ovnkube-node-2db6t"
Jul 16 11:03:31.300469 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.298305    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/69aed169-e8af-429d-99c3-3b20be64e351-ovnkube-script-lib\") pod \"ovnkube-node-2db6t\" (UID: \"69aed169-e8af-429d-99c3-3b20be64e351\") " pod="openshift-ovn-kubernetes/ovnkube-node-2db6t"
Jul 16 11:03:31.300469 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.298780    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8de3fcaf-5b76-4d0f-8453-60b892832242-etc-tuned\") pod \"tuned-cr2x7\" (UID: \"8de3fcaf-5b76-4d0f-8453-60b892832242\") " pod="openshift-cluster-node-tuning-operator/tuned-cr2x7"
Jul 16 11:03:31.300469 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.299255    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8de3fcaf-5b76-4d0f-8453-60b892832242-tmp\") pod \"tuned-cr2x7\" (UID: \"8de3fcaf-5b76-4d0f-8453-60b892832242\") " pod="openshift-cluster-node-tuning-operator/tuned-cr2x7"
Jul 16 11:03:31.300469 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.300313    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/69aed169-e8af-429d-99c3-3b20be64e351-ovn-node-metrics-cert\") pod \"ovnkube-node-2db6t\" (UID: \"69aed169-e8af-429d-99c3-3b20be64e351\") " pod="openshift-ovn-kubernetes/ovnkube-node-2db6t"
Jul 16 11:03:31.311563 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.311479    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zwq75\" (UniqueName: \"kubernetes.io/projected/a3d44785-92d8-4a5e-bc2e-b91f4fa8b509-kube-api-access-zwq75\") pod \"node-resolver-z5dtv\" (UID: \"a3d44785-92d8-4a5e-bc2e-b91f4fa8b509\") " pod="openshift-dns/node-resolver-z5dtv"
Jul 16 11:03:31.314313 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:03:31.314267    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 16 11:03:31.314313 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:03:31.314298    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 16 11:03:31.314313 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:03:31.314313    2583 projected.go:196] Error preparing data for projected volume kube-api-access-2kcc2 for pod openshift-network-diagnostics/network-check-target-rc7w4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 11:03:31.314656 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:03:31.314394    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3d0ee8e8-4018-4963-b553-af9b6ecb0f80-kube-api-access-2kcc2 podName:3d0ee8e8-4018-4963-b553-af9b6ecb0f80 nodeName:}" failed. No retries permitted until 2026-07-16 11:03:31.814374526 +0000 UTC m=+4.448595960 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-2kcc2" (UniqueName: "kubernetes.io/projected/3d0ee8e8-4018-4963-b553-af9b6ecb0f80-kube-api-access-2kcc2") pod "network-check-target-rc7w4" (UID: "3d0ee8e8-4018-4963-b553-af9b6ecb0f80") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 11:03:31.316746 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.316710    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6rq47\" (UniqueName: \"kubernetes.io/projected/69aed169-e8af-429d-99c3-3b20be64e351-kube-api-access-6rq47\") pod \"ovnkube-node-2db6t\" (UID: \"69aed169-e8af-429d-99c3-3b20be64e351\") " pod="openshift-ovn-kubernetes/ovnkube-node-2db6t"
Jul 16 11:03:31.316926 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.316901    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qb2ts\" (UniqueName: \"kubernetes.io/projected/6fd1e802-78a8-4991-99ee-47cfea84ab62-kube-api-access-qb2ts\") pod \"multus-additional-cni-plugins-8r89z\" (UID: \"6fd1e802-78a8-4991-99ee-47cfea84ab62\") " pod="openshift-multus/multus-additional-cni-plugins-8r89z"
Jul 16 11:03:31.316991 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.316901    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7m82s\" (UniqueName: \"kubernetes.io/projected/8de3fcaf-5b76-4d0f-8453-60b892832242-kube-api-access-7m82s\") pod \"tuned-cr2x7\" (UID: \"8de3fcaf-5b76-4d0f-8453-60b892832242\") " pod="openshift-cluster-node-tuning-operator/tuned-cr2x7"
Jul 16 11:03:31.317313 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.317290    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vcz49\" (UniqueName: \"kubernetes.io/projected/0244f4b4-dd39-4870-97be-8378a99c93a2-kube-api-access-vcz49\") pod \"node-ca-6xfs6\" (UID: \"0244f4b4-dd39-4870-97be-8378a99c93a2\") " pod="openshift-image-registry/node-ca-6xfs6"
Jul 16 11:03:31.387166 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.387127    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wtqvh"
Jul 16 11:03:31.395844 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.395811    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-zs9vs"
Jul 16 11:03:31.406248 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.406211    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-x6lfk"
Jul 16 11:03:31.413570 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.413540    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-qxpqt"
Jul 16 11:03:31.420833 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.420808    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-cr2x7"
Jul 16 11:03:31.428878 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.428845    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-6xfs6"
Jul 16 11:03:31.436079 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.436045    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-8r89z"
Jul 16 11:03:31.442434 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.442405    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-z5dtv"
Jul 16 11:03:31.447718 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.447690    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-2db6t"
Jul 16 11:03:31.517474 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:31.517424    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc24aaddd_b44c_4155_a7c5_fadb281f9b39.slice/crio-9d5502e81afdd5bacc5097c819e7de0d1a39a5c79ca62ab866c2d71315722010 WatchSource:0}: Error finding container 9d5502e81afdd5bacc5097c819e7de0d1a39a5c79ca62ab866c2d71315722010: Status 404 returned error can't find the container with id 9d5502e81afdd5bacc5097c819e7de0d1a39a5c79ca62ab866c2d71315722010
Jul 16 11:03:31.518506 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:31.518476    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8de3fcaf_5b76_4d0f_8453_60b892832242.slice/crio-3c2257ff898cbb7e18fb841da354ab04fe406df07c519f7e1977eb9c379a0e19 WatchSource:0}: Error finding container 3c2257ff898cbb7e18fb841da354ab04fe406df07c519f7e1977eb9c379a0e19: Status 404 returned error can't find the container with id 3c2257ff898cbb7e18fb841da354ab04fe406df07c519f7e1977eb9c379a0e19
Jul 16 11:03:31.521949 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:31.521924    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod69aed169_e8af_429d_99c3_3b20be64e351.slice/crio-38c41cdac0c16757e481b04d1a4e7449c477a589cb789e945df848a04a5d6cf0 WatchSource:0}: Error finding container 38c41cdac0c16757e481b04d1a4e7449c477a589cb789e945df848a04a5d6cf0: Status 404 returned error can't find the container with id 38c41cdac0c16757e481b04d1a4e7449c477a589cb789e945df848a04a5d6cf0
Jul 16 11:03:31.525072 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:31.525045    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6fd1e802_78a8_4991_99ee_47cfea84ab62.slice/crio-27c8ac797c8b777adb5183a9a15b4a831e24c03c99a180eca9aa0257cb7159ee WatchSource:0}: Error finding container 27c8ac797c8b777adb5183a9a15b4a831e24c03c99a180eca9aa0257cb7159ee: Status 404 returned error can't find the container with id 27c8ac797c8b777adb5183a9a15b4a831e24c03c99a180eca9aa0257cb7159ee
Jul 16 11:03:31.525623 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:31.525595    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3d44785_92d8_4a5e_bc2e_b91f4fa8b509.slice/crio-3bc66ba3327448cb77bfa400dd57b43399c8ae1d8b46f7b19039326bbff733cb WatchSource:0}: Error finding container 3bc66ba3327448cb77bfa400dd57b43399c8ae1d8b46f7b19039326bbff733cb: Status 404 returned error can't find the container with id 3bc66ba3327448cb77bfa400dd57b43399c8ae1d8b46f7b19039326bbff733cb
Jul 16 11:03:31.528858 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:31.527495    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5baff3f6_85da_46fb_b42b_2e70e7d77cc4.slice/crio-90437d5338a58d9457e25edff13739b486138b351e9befbacefeedf2caefc1c5 WatchSource:0}: Error finding container 90437d5338a58d9457e25edff13739b486138b351e9befbacefeedf2caefc1c5: Status 404 returned error can't find the container with id 90437d5338a58d9457e25edff13739b486138b351e9befbacefeedf2caefc1c5
Jul 16 11:03:31.529302 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:31.529013    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0244f4b4_dd39_4870_97be_8378a99c93a2.slice/crio-366a3414ab73588758cef3b4f98b6715145bfa38fe9b9a6bf89fd8e81ee6bc23 WatchSource:0}: Error finding container 366a3414ab73588758cef3b4f98b6715145bfa38fe9b9a6bf89fd8e81ee6bc23: Status 404 returned error can't find the container with id 366a3414ab73588758cef3b4f98b6715145bfa38fe9b9a6bf89fd8e81ee6bc23
Jul 16 11:03:31.531300 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:31.530818    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d4103af_f06a_4592_83e0_8625e6267267.slice/crio-6bb66c388cf073049f5df678648010a87a862864a319157fccb1bf6b075c5095 WatchSource:0}: Error finding container 6bb66c388cf073049f5df678648010a87a862864a319157fccb1bf6b075c5095: Status 404 returned error can't find the container with id 6bb66c388cf073049f5df678648010a87a862864a319157fccb1bf6b075c5095
Jul 16 11:03:31.531859 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:03:31.531834    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod21e8d664_4b0d_45cb_914f_7f55bb26d1f0.slice/crio-3e79b5d5d749cdab6a71bb4cc3001118c2b0b75d66ba65141e94044f0502ce2c WatchSource:0}: Error finding container 3e79b5d5d749cdab6a71bb4cc3001118c2b0b75d66ba65141e94044f0502ce2c: Status 404 returned error can't find the container with id 3e79b5d5d749cdab6a71bb4cc3001118c2b0b75d66ba65141e94044f0502ce2c
Jul 16 11:03:31.701822 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.701704    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d4e7c597-9836-4a29-8591-e56938995710-metrics-certs\") pod \"network-metrics-daemon-qcl5d\" (UID: \"d4e7c597-9836-4a29-8591-e56938995710\") " pod="openshift-multus/network-metrics-daemon-qcl5d"
Jul 16 11:03:31.701997 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:03:31.701822    2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 11:03:31.701997 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:03:31.701896    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d4e7c597-9836-4a29-8591-e56938995710-metrics-certs podName:d4e7c597-9836-4a29-8591-e56938995710 nodeName:}" failed. No retries permitted until 2026-07-16 11:03:32.701880164 +0000 UTC m=+5.336101598 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d4e7c597-9836-4a29-8591-e56938995710-metrics-certs") pod "network-metrics-daemon-qcl5d" (UID: "d4e7c597-9836-4a29-8591-e56938995710") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 11:03:31.802581 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.802538    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c88b6ce0-4926-4de9-942e-7f1b30c33e9e-original-pull-secret\") pod \"global-pull-secret-syncer-hlm2s\" (UID: \"c88b6ce0-4926-4de9-942e-7f1b30c33e9e\") " pod="kube-system/global-pull-secret-syncer-hlm2s"
Jul 16 11:03:31.802813 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:03:31.802709    2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 16 11:03:31.802813 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:03:31.802794    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c88b6ce0-4926-4de9-942e-7f1b30c33e9e-original-pull-secret podName:c88b6ce0-4926-4de9-942e-7f1b30c33e9e nodeName:}" failed. No retries permitted until 2026-07-16 11:03:32.802771091 +0000 UTC m=+5.436992538 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c88b6ce0-4926-4de9-942e-7f1b30c33e9e-original-pull-secret") pod "global-pull-secret-syncer-hlm2s" (UID: "c88b6ce0-4926-4de9-942e-7f1b30c33e9e") : object "kube-system"/"original-pull-secret" not registered
Jul 16 11:03:31.903787 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:31.903739    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2kcc2\" (UniqueName: \"kubernetes.io/projected/3d0ee8e8-4018-4963-b553-af9b6ecb0f80-kube-api-access-2kcc2\") pod \"network-check-target-rc7w4\" (UID: \"3d0ee8e8-4018-4963-b553-af9b6ecb0f80\") " pod="openshift-network-diagnostics/network-check-target-rc7w4"
Jul 16 11:03:31.903958 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:03:31.903879    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 16 11:03:31.903958 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:03:31.903905    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 16 11:03:31.903958 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:03:31.903917    2583 projected.go:196] Error preparing data for projected volume kube-api-access-2kcc2 for pod openshift-network-diagnostics/network-check-target-rc7w4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 11:03:31.904071 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:03:31.903981    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3d0ee8e8-4018-4963-b553-af9b6ecb0f80-kube-api-access-2kcc2 podName:3d0ee8e8-4018-4963-b553-af9b6ecb0f80 nodeName:}" failed. No retries permitted until 2026-07-16 11:03:32.903963204 +0000 UTC m=+5.538184652 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-2kcc2" (UniqueName: "kubernetes.io/projected/3d0ee8e8-4018-4963-b553-af9b6ecb0f80-kube-api-access-2kcc2") pod "network-check-target-rc7w4" (UID: "3d0ee8e8-4018-4963-b553-af9b6ecb0f80") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 11:03:32.201332 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:32.200944    2583 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-15 10:58:31 +0000 UTC" deadline="2027-12-22 01:42:08.652293843 +0000 UTC"
Jul 16 11:03:32.201332 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:32.201264    2583 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12566h38m36.451037455s"
Jul 16 11:03:32.223963 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:32.223388    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hlm2s"
Jul 16 11:03:32.223963 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:03:32.223532    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hlm2s" podUID="c88b6ce0-4926-4de9-942e-7f1b30c33e9e"
Jul 16 11:03:32.265844 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:32.265778    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-zs9vs" event={"ID":"5baff3f6-85da-46fb-b42b-2e70e7d77cc4","Type":"ContainerStarted","Data":"90437d5338a58d9457e25edff13739b486138b351e9befbacefeedf2caefc1c5"}
Jul 16 11:03:32.278576 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:32.278474    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-z5dtv" event={"ID":"a3d44785-92d8-4a5e-bc2e-b91f4fa8b509","Type":"ContainerStarted","Data":"3bc66ba3327448cb77bfa400dd57b43399c8ae1d8b46f7b19039326bbff733cb"}
Jul 16 11:03:32.287651 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:32.287344    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8r89z" event={"ID":"6fd1e802-78a8-4991-99ee-47cfea84ab62","Type":"ContainerStarted","Data":"27c8ac797c8b777adb5183a9a15b4a831e24c03c99a180eca9aa0257cb7159ee"}
Jul 16 11:03:32.309823 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:32.309779    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-cr2x7" event={"ID":"8de3fcaf-5b76-4d0f-8453-60b892832242","Type":"ContainerStarted","Data":"3c2257ff898cbb7e18fb841da354ab04fe406df07c519f7e1977eb9c379a0e19"}
Jul 16 11:03:32.323697 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:32.323621    2583 generic.go:356] "Generic (PLEG): container finished" podID="b31dba48238fdc286bfab6bd4db2f24f" containerID="0ad3d23d83d75f270c4cad08c86781cb141cfcaeefb401e9ff0bc07e8fb8bdfb" exitCode=0
Jul 16 11:03:32.323896 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:32.323764    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-28.ec2.internal" event={"ID":"b31dba48238fdc286bfab6bd4db2f24f","Type":"ContainerDied","Data":"0ad3d23d83d75f270c4cad08c86781cb141cfcaeefb401e9ff0bc07e8fb8bdfb"}
Jul 16 11:03:32.341143 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:32.341041    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-28.ec2.internal" event={"ID":"43ba1ebae5d7ee21a4751f58c11fde6b","Type":"ContainerStarted","Data":"b4e61fd9c845311d74f9af6f71eb96e54df5e75fee44152250c21532dcacafc4"}
Jul 16 11:03:32.364787 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:32.364443    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wtqvh" event={"ID":"21e8d664-4b0d-45cb-914f-7f55bb26d1f0","Type":"ContainerStarted","Data":"3e79b5d5d749cdab6a71bb4cc3001118c2b0b75d66ba65141e94044f0502ce2c"}
Jul 16 11:03:32.384022 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:32.383881    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-x6lfk" event={"ID":"2d4103af-f06a-4592-83e0-8625e6267267","Type":"ContainerStarted","Data":"6bb66c388cf073049f5df678648010a87a862864a319157fccb1bf6b075c5095"}
Jul 16 11:03:32.393718 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:32.393660    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-6xfs6" event={"ID":"0244f4b4-dd39-4870-97be-8378a99c93a2","Type":"ContainerStarted","Data":"366a3414ab73588758cef3b4f98b6715145bfa38fe9b9a6bf89fd8e81ee6bc23"}
Jul 16 11:03:32.412100 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:32.412060    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2db6t" event={"ID":"69aed169-e8af-429d-99c3-3b20be64e351","Type":"ContainerStarted","Data":"38c41cdac0c16757e481b04d1a4e7449c477a589cb789e945df848a04a5d6cf0"}
Jul 16 11:03:32.416544 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:32.416505    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-qxpqt" event={"ID":"c24aaddd-b44c-4155-a7c5-fadb281f9b39","Type":"ContainerStarted","Data":"9d5502e81afdd5bacc5097c819e7de0d1a39a5c79ca62ab866c2d71315722010"}
Jul 16 11:03:32.712385 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:32.712344    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d4e7c597-9836-4a29-8591-e56938995710-metrics-certs\") pod \"network-metrics-daemon-qcl5d\" (UID: \"d4e7c597-9836-4a29-8591-e56938995710\") " pod="openshift-multus/network-metrics-daemon-qcl5d"
Jul 16 11:03:32.712600 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:03:32.712528    2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 11:03:32.712716 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:03:32.712605    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d4e7c597-9836-4a29-8591-e56938995710-metrics-certs podName:d4e7c597-9836-4a29-8591-e56938995710 nodeName:}" failed. No retries permitted until 2026-07-16 11:03:34.712582313 +0000 UTC m=+7.346803774 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d4e7c597-9836-4a29-8591-e56938995710-metrics-certs") pod "network-metrics-daemon-qcl5d" (UID: "d4e7c597-9836-4a29-8591-e56938995710") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 11:03:32.793684 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:32.793372    2583 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 16 11:03:32.813119 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:32.813081    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c88b6ce0-4926-4de9-942e-7f1b30c33e9e-original-pull-secret\") pod \"global-pull-secret-syncer-hlm2s\" (UID: \"c88b6ce0-4926-4de9-942e-7f1b30c33e9e\") " pod="kube-system/global-pull-secret-syncer-hlm2s"
Jul 16 11:03:32.813319 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:03:32.813292    2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 16 11:03:32.813380 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:03:32.813360    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c88b6ce0-4926-4de9-942e-7f1b30c33e9e-original-pull-secret podName:c88b6ce0-4926-4de9-942e-7f1b30c33e9e nodeName:}" failed. No retries permitted until 2026-07-16 11:03:34.813340441 +0000 UTC m=+7.447561881 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c88b6ce0-4926-4de9-942e-7f1b30c33e9e-original-pull-secret") pod "global-pull-secret-syncer-hlm2s" (UID: "c88b6ce0-4926-4de9-942e-7f1b30c33e9e") : object "kube-system"/"original-pull-secret" not registered
Jul 16 11:03:32.914149 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:32.914105    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2kcc2\" (UniqueName: \"kubernetes.io/projected/3d0ee8e8-4018-4963-b553-af9b6ecb0f80-kube-api-access-2kcc2\") pod \"network-check-target-rc7w4\" (UID: \"3d0ee8e8-4018-4963-b553-af9b6ecb0f80\") " pod="openshift-network-diagnostics/network-check-target-rc7w4"
Jul 16 11:03:32.914380 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:03:32.914360    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 16 11:03:32.914463 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:03:32.914387    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 16 11:03:32.914463 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:03:32.914401    2583 projected.go:196] Error preparing data for projected volume kube-api-access-2kcc2 for pod openshift-network-diagnostics/network-check-target-rc7w4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 11:03:32.914463 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:03:32.914462    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3d0ee8e8-4018-4963-b553-af9b6ecb0f80-kube-api-access-2kcc2 podName:3d0ee8e8-4018-4963-b553-af9b6ecb0f80 nodeName:}" failed. No retries permitted until 2026-07-16 11:03:34.914443371 +0000 UTC m=+7.548664811 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-2kcc2" (UniqueName: "kubernetes.io/projected/3d0ee8e8-4018-4963-b553-af9b6ecb0f80-kube-api-access-2kcc2") pod "network-check-target-rc7w4" (UID: "3d0ee8e8-4018-4963-b553-af9b6ecb0f80") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 11:03:33.202390 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:33.202256    2583 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-15 10:58:31 +0000 UTC" deadline="2027-12-20 00:07:21.558037854 +0000 UTC"
Jul 16 11:03:33.202390 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:33.202299    2583 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12517h3m48.355744529s"
Jul 16 11:03:33.225401 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:33.223761    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qcl5d"
Jul 16 11:03:33.225401 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:03:33.223928    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qcl5d" podUID="d4e7c597-9836-4a29-8591-e56938995710"
Jul 16 11:03:33.225401 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:33.224471    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rc7w4"
Jul 16 11:03:33.225401 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:03:33.224584    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rc7w4" podUID="3d0ee8e8-4018-4963-b553-af9b6ecb0f80"
Jul 16 11:03:33.441420 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:33.441384    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-28.ec2.internal" event={"ID":"b31dba48238fdc286bfab6bd4db2f24f","Type":"ContainerStarted","Data":"5ffdae5f3eb1237e3dd1d2eeb622e5f31451f12ddccc2ebcd40a617f14ec5ea5"}
Jul 16 11:03:34.222617 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:34.222571    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hlm2s"
Jul 16 11:03:34.223180 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:03:34.222751    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hlm2s" podUID="c88b6ce0-4926-4de9-942e-7f1b30c33e9e"
Jul 16 11:03:34.730814 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:34.730772    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d4e7c597-9836-4a29-8591-e56938995710-metrics-certs\") pod \"network-metrics-daemon-qcl5d\" (UID: \"d4e7c597-9836-4a29-8591-e56938995710\") " pod="openshift-multus/network-metrics-daemon-qcl5d"
Jul 16 11:03:34.731019 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:03:34.730941    2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 11:03:34.731019 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:03:34.731012    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d4e7c597-9836-4a29-8591-e56938995710-metrics-certs podName:d4e7c597-9836-4a29-8591-e56938995710 nodeName:}" failed. No retries permitted until 2026-07-16 11:03:38.730991996 +0000 UTC m=+11.365213444 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d4e7c597-9836-4a29-8591-e56938995710-metrics-certs") pod "network-metrics-daemon-qcl5d" (UID: "d4e7c597-9836-4a29-8591-e56938995710") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 11:03:34.831331 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:34.831286    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c88b6ce0-4926-4de9-942e-7f1b30c33e9e-original-pull-secret\") pod \"global-pull-secret-syncer-hlm2s\" (UID: \"c88b6ce0-4926-4de9-942e-7f1b30c33e9e\") " pod="kube-system/global-pull-secret-syncer-hlm2s"
Jul 16 11:03:34.831523 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:03:34.831467    2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 16 11:03:34.831600 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:03:34.831532    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c88b6ce0-4926-4de9-942e-7f1b30c33e9e-original-pull-secret podName:c88b6ce0-4926-4de9-942e-7f1b30c33e9e nodeName:}" failed. No retries permitted until 2026-07-16 11:03:38.831512884 +0000 UTC m=+11.465734335 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c88b6ce0-4926-4de9-942e-7f1b30c33e9e-original-pull-secret") pod "global-pull-secret-syncer-hlm2s" (UID: "c88b6ce0-4926-4de9-942e-7f1b30c33e9e") : object "kube-system"/"original-pull-secret" not registered
Jul 16 11:03:34.932567 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:34.932525    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2kcc2\" (UniqueName: \"kubernetes.io/projected/3d0ee8e8-4018-4963-b553-af9b6ecb0f80-kube-api-access-2kcc2\") pod \"network-check-target-rc7w4\" (UID: \"3d0ee8e8-4018-4963-b553-af9b6ecb0f80\") " pod="openshift-network-diagnostics/network-check-target-rc7w4"
Jul 16 11:03:34.932785 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:03:34.932666    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 16 11:03:34.932785 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:03:34.932691    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 16 11:03:34.932785 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:03:34.932707    2583 projected.go:196] Error preparing data for projected volume kube-api-access-2kcc2 for pod openshift-network-diagnostics/network-check-target-rc7w4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 11:03:34.932785 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:03:34.932773    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3d0ee8e8-4018-4963-b553-af9b6ecb0f80-kube-api-access-2kcc2 podName:3d0ee8e8-4018-4963-b553-af9b6ecb0f80 nodeName:}" failed. No retries permitted until 2026-07-16 11:03:38.932754893 +0000 UTC m=+11.566976343 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-2kcc2" (UniqueName: "kubernetes.io/projected/3d0ee8e8-4018-4963-b553-af9b6ecb0f80-kube-api-access-2kcc2") pod "network-check-target-rc7w4" (UID: "3d0ee8e8-4018-4963-b553-af9b6ecb0f80") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 11:03:35.223135 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:35.222968    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qcl5d"
Jul 16 11:03:35.223135 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:35.222991    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rc7w4"
Jul 16 11:03:35.223640 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:03:35.223136    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qcl5d" podUID="d4e7c597-9836-4a29-8591-e56938995710"
Jul 16 11:03:35.223640 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:03:35.223579    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rc7w4" podUID="3d0ee8e8-4018-4963-b553-af9b6ecb0f80"
Jul 16 11:03:36.222589 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:36.222549    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hlm2s"
Jul 16 11:03:36.222817 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:03:36.222709    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hlm2s" podUID="c88b6ce0-4926-4de9-942e-7f1b30c33e9e"
Jul 16 11:03:37.223322 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:37.222756    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qcl5d"
Jul 16 11:03:37.223322 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:03:37.222925    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qcl5d" podUID="d4e7c597-9836-4a29-8591-e56938995710"
Jul 16 11:03:37.223322 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:37.223319    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rc7w4"
Jul 16 11:03:37.223965 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:03:37.223412    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rc7w4" podUID="3d0ee8e8-4018-4963-b553-af9b6ecb0f80"
Jul 16 11:03:38.223266 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:38.223222    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hlm2s"
Jul 16 11:03:38.223479 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:03:38.223384    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hlm2s" podUID="c88b6ce0-4926-4de9-942e-7f1b30c33e9e"
Jul 16 11:03:38.768409 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:38.768315    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d4e7c597-9836-4a29-8591-e56938995710-metrics-certs\") pod \"network-metrics-daemon-qcl5d\" (UID: \"d4e7c597-9836-4a29-8591-e56938995710\") " pod="openshift-multus/network-metrics-daemon-qcl5d"
Jul 16 11:03:38.768622 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:03:38.768486    2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 11:03:38.768622 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:03:38.768558    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d4e7c597-9836-4a29-8591-e56938995710-metrics-certs podName:d4e7c597-9836-4a29-8591-e56938995710 nodeName:}" failed. No retries permitted until 2026-07-16 11:03:46.768539649 +0000 UTC m=+19.402761101 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d4e7c597-9836-4a29-8591-e56938995710-metrics-certs") pod "network-metrics-daemon-qcl5d" (UID: "d4e7c597-9836-4a29-8591-e56938995710") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 11:03:38.869498 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:38.869462    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c88b6ce0-4926-4de9-942e-7f1b30c33e9e-original-pull-secret\") pod \"global-pull-secret-syncer-hlm2s\" (UID: \"c88b6ce0-4926-4de9-942e-7f1b30c33e9e\") " pod="kube-system/global-pull-secret-syncer-hlm2s"
Jul 16 11:03:38.869719 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:03:38.869623    2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 16 11:03:38.869719 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:03:38.869689    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c88b6ce0-4926-4de9-942e-7f1b30c33e9e-original-pull-secret podName:c88b6ce0-4926-4de9-942e-7f1b30c33e9e nodeName:}" failed. No retries permitted until 2026-07-16 11:03:46.869670804 +0000 UTC m=+19.503892237 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c88b6ce0-4926-4de9-942e-7f1b30c33e9e-original-pull-secret") pod "global-pull-secret-syncer-hlm2s" (UID: "c88b6ce0-4926-4de9-942e-7f1b30c33e9e") : object "kube-system"/"original-pull-secret" not registered
Jul 16 11:03:38.970354 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:38.970312    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2kcc2\" (UniqueName: \"kubernetes.io/projected/3d0ee8e8-4018-4963-b553-af9b6ecb0f80-kube-api-access-2kcc2\") pod \"network-check-target-rc7w4\" (UID: \"3d0ee8e8-4018-4963-b553-af9b6ecb0f80\") " pod="openshift-network-diagnostics/network-check-target-rc7w4"
Jul 16 11:03:38.971001 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:03:38.970528    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 16 11:03:38.971001 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:03:38.970553    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 16 11:03:38.971001 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:03:38.970566    2583 projected.go:196] Error preparing data for projected volume kube-api-access-2kcc2 for pod openshift-network-diagnostics/network-check-target-rc7w4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 11:03:38.971001 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:03:38.970652    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3d0ee8e8-4018-4963-b553-af9b6ecb0f80-kube-api-access-2kcc2 podName:3d0ee8e8-4018-4963-b553-af9b6ecb0f80 nodeName:}" failed. No retries permitted until 2026-07-16 11:03:46.970629575 +0000 UTC m=+19.604851018 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-2kcc2" (UniqueName: "kubernetes.io/projected/3d0ee8e8-4018-4963-b553-af9b6ecb0f80-kube-api-access-2kcc2") pod "network-check-target-rc7w4" (UID: "3d0ee8e8-4018-4963-b553-af9b6ecb0f80") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 11:03:39.224833 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:39.224109    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rc7w4"
Jul 16 11:03:39.224833 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:03:39.224237    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rc7w4" podUID="3d0ee8e8-4018-4963-b553-af9b6ecb0f80"
Jul 16 11:03:39.224833 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:39.224653    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qcl5d"
Jul 16 11:03:39.224833 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:03:39.224749    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qcl5d" podUID="d4e7c597-9836-4a29-8591-e56938995710"
Jul 16 11:03:40.223045 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:40.223008    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hlm2s"
Jul 16 11:03:40.223236 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:03:40.223139    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hlm2s" podUID="c88b6ce0-4926-4de9-942e-7f1b30c33e9e"
Jul 16 11:03:41.223051 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:41.223003    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rc7w4"
Jul 16 11:03:41.223544 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:41.223003    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qcl5d"
Jul 16 11:03:41.223544 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:03:41.223152    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rc7w4" podUID="3d0ee8e8-4018-4963-b553-af9b6ecb0f80"
Jul 16 11:03:41.223544 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:03:41.223234    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qcl5d" podUID="d4e7c597-9836-4a29-8591-e56938995710"
Jul 16 11:03:42.222912 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:42.222871    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hlm2s"
Jul 16 11:03:42.223084 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:03:42.223010    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hlm2s" podUID="c88b6ce0-4926-4de9-942e-7f1b30c33e9e"
Jul 16 11:03:43.223449 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:43.223411    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qcl5d"
Jul 16 11:03:43.223911 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:43.223411    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rc7w4"
Jul 16 11:03:43.223911 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:03:43.223570    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qcl5d" podUID="d4e7c597-9836-4a29-8591-e56938995710"
Jul 16 11:03:43.223911 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:03:43.223629    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rc7w4" podUID="3d0ee8e8-4018-4963-b553-af9b6ecb0f80"
Jul 16 11:03:44.222652 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:44.222610    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hlm2s"
Jul 16 11:03:44.222839 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:03:44.222731    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hlm2s" podUID="c88b6ce0-4926-4de9-942e-7f1b30c33e9e"
Jul 16 11:03:45.222856 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:45.222810    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qcl5d"
Jul 16 11:03:45.223318 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:03:45.222960    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qcl5d" podUID="d4e7c597-9836-4a29-8591-e56938995710"
Jul 16 11:03:45.223318 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:45.222814    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rc7w4"
Jul 16 11:03:45.223421 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:03:45.223388    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rc7w4" podUID="3d0ee8e8-4018-4963-b553-af9b6ecb0f80"
Jul 16 11:03:46.222846 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:46.222806    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hlm2s"
Jul 16 11:03:46.223100 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:03:46.222948    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hlm2s" podUID="c88b6ce0-4926-4de9-942e-7f1b30c33e9e"
Jul 16 11:03:46.829520 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:46.829471    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d4e7c597-9836-4a29-8591-e56938995710-metrics-certs\") pod \"network-metrics-daemon-qcl5d\" (UID: \"d4e7c597-9836-4a29-8591-e56938995710\") " pod="openshift-multus/network-metrics-daemon-qcl5d"
Jul 16 11:03:46.829731 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:03:46.829655    2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 11:03:46.829796 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:03:46.829740    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d4e7c597-9836-4a29-8591-e56938995710-metrics-certs podName:d4e7c597-9836-4a29-8591-e56938995710 nodeName:}" failed. No retries permitted until 2026-07-16 11:04:02.829717614 +0000 UTC m=+35.463939076 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d4e7c597-9836-4a29-8591-e56938995710-metrics-certs") pod "network-metrics-daemon-qcl5d" (UID: "d4e7c597-9836-4a29-8591-e56938995710") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 11:03:46.930437 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:46.930394    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c88b6ce0-4926-4de9-942e-7f1b30c33e9e-original-pull-secret\") pod \"global-pull-secret-syncer-hlm2s\" (UID: \"c88b6ce0-4926-4de9-942e-7f1b30c33e9e\") " pod="kube-system/global-pull-secret-syncer-hlm2s"
Jul 16 11:03:46.930638 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:03:46.930557    2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 16 11:03:46.930681 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:03:46.930650    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c88b6ce0-4926-4de9-942e-7f1b30c33e9e-original-pull-secret podName:c88b6ce0-4926-4de9-942e-7f1b30c33e9e nodeName:}" failed. No retries permitted until 2026-07-16 11:04:02.930634951 +0000 UTC m=+35.564856387 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c88b6ce0-4926-4de9-942e-7f1b30c33e9e-original-pull-secret") pod "global-pull-secret-syncer-hlm2s" (UID: "c88b6ce0-4926-4de9-942e-7f1b30c33e9e") : object "kube-system"/"original-pull-secret" not registered
Jul 16 11:03:47.031838 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:47.031792    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2kcc2\" (UniqueName: \"kubernetes.io/projected/3d0ee8e8-4018-4963-b553-af9b6ecb0f80-kube-api-access-2kcc2\") pod \"network-check-target-rc7w4\" (UID: \"3d0ee8e8-4018-4963-b553-af9b6ecb0f80\") " pod="openshift-network-diagnostics/network-check-target-rc7w4"
Jul 16 11:03:47.032032 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:03:47.031976    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 16 11:03:47.032032 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:03:47.032005    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 16 11:03:47.032032 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:03:47.032018    2583 projected.go:196] Error preparing data for projected volume kube-api-access-2kcc2 for pod openshift-network-diagnostics/network-check-target-rc7w4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 11:03:47.032238 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:03:47.032074    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3d0ee8e8-4018-4963-b553-af9b6ecb0f80-kube-api-access-2kcc2 podName:3d0ee8e8-4018-4963-b553-af9b6ecb0f80 nodeName:}" failed. No retries permitted until 2026-07-16 11:04:03.032059134 +0000 UTC m=+35.666280582 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-2kcc2" (UniqueName: "kubernetes.io/projected/3d0ee8e8-4018-4963-b553-af9b6ecb0f80-kube-api-access-2kcc2") pod "network-check-target-rc7w4" (UID: "3d0ee8e8-4018-4963-b553-af9b6ecb0f80") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 11:03:47.223368 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:47.223291    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qcl5d"
Jul 16 11:03:47.223803 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:03:47.223409    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qcl5d" podUID="d4e7c597-9836-4a29-8591-e56938995710"
Jul 16 11:03:47.223803 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:47.223472    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rc7w4"
Jul 16 11:03:47.223803 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:03:47.223582    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rc7w4" podUID="3d0ee8e8-4018-4963-b553-af9b6ecb0f80"
Jul 16 11:03:48.222795 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:48.222759    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hlm2s"
Jul 16 11:03:48.222992 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:03:48.222894    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hlm2s" podUID="c88b6ce0-4926-4de9-942e-7f1b30c33e9e"
Jul 16 11:03:49.224062 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:49.224029    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qcl5d"
Jul 16 11:03:49.224518 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:03:49.224160    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qcl5d" podUID="d4e7c597-9836-4a29-8591-e56938995710"
Jul 16 11:03:49.224518 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:49.224204    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rc7w4"
Jul 16 11:03:49.224518 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:03:49.224328    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rc7w4" podUID="3d0ee8e8-4018-4963-b553-af9b6ecb0f80"
Jul 16 11:03:50.223022 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:50.222969    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hlm2s"
Jul 16 11:03:50.223212 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:03:50.223098    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hlm2s" podUID="c88b6ce0-4926-4de9-942e-7f1b30c33e9e"
Jul 16 11:03:51.223560 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:51.223518    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qcl5d"
Jul 16 11:03:51.224141 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:03:51.223686    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qcl5d" podUID="d4e7c597-9836-4a29-8591-e56938995710"
Jul 16 11:03:51.224141 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:51.223707    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rc7w4"
Jul 16 11:03:51.224141 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:03:51.223793    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rc7w4" podUID="3d0ee8e8-4018-4963-b553-af9b6ecb0f80"
Jul 16 11:03:52.223021 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:52.222656    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hlm2s"
Jul 16 11:03:52.223185 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:03:52.223135    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hlm2s" podUID="c88b6ce0-4926-4de9-942e-7f1b30c33e9e"
Jul 16 11:03:52.492768 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:52.492725    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-28.ec2.internal" event={"ID":"43ba1ebae5d7ee21a4751f58c11fde6b","Type":"ContainerStarted","Data":"21242650fa2579fa7f0164d06c6530f5cd090b5f0a1df3fe0f8830e4fde1ebfb"}
Jul 16 11:03:52.494512 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:52.494487    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wtqvh" event={"ID":"21e8d664-4b0d-45cb-914f-7f55bb26d1f0","Type":"ContainerStarted","Data":"6397c83c097d086e406ea8ae3ed69eafc6faa27863dfc1f6e875940e2617ac6d"}
Jul 16 11:03:52.496185 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:52.496068    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-6xfs6" event={"ID":"0244f4b4-dd39-4870-97be-8378a99c93a2","Type":"ContainerStarted","Data":"81d3b15a17de19de6fff15f21731f4cc101c3f9c82bc17839ae39ae550926a55"}
Jul 16 11:03:52.499260 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:52.499242    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2db6t_69aed169-e8af-429d-99c3-3b20be64e351/ovn-acl-logging/0.log"
Jul 16 11:03:52.499678 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:52.499650    2583 generic.go:356] "Generic (PLEG): container finished" podID="69aed169-e8af-429d-99c3-3b20be64e351" containerID="dddf4eda2f0cba7e6db049d69dae8f5fca0c25f385573de83fc4a6a76dad2af5" exitCode=1
Jul 16 11:03:52.499770 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:52.499715    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2db6t" event={"ID":"69aed169-e8af-429d-99c3-3b20be64e351","Type":"ContainerStarted","Data":"ea029fa01e2d2f7825dc1ec0d7e3195df647f0dbb8bf5343e471d06c02021347"}
Jul 16 11:03:52.499844 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:52.499800    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2db6t" event={"ID":"69aed169-e8af-429d-99c3-3b20be64e351","Type":"ContainerStarted","Data":"2806d4e8bb0738451d99273c1d71a6c32a1d2b215270ab4988ff8a818e7556de"}
Jul 16 11:03:52.499896 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:52.499854    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2db6t" event={"ID":"69aed169-e8af-429d-99c3-3b20be64e351","Type":"ContainerStarted","Data":"94cbd54dc90ed8e37667421c946bae66d08fcc608f90dab8aec762b7a5eb12a6"}
Jul 16 11:03:52.499896 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:52.499885    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2db6t" event={"ID":"69aed169-e8af-429d-99c3-3b20be64e351","Type":"ContainerStarted","Data":"49d00bf59a8ad817d4df0698a569a00467cab5b61a2034c3bd7a68467ac8083f"}
Jul 16 11:03:52.499987 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:52.499926    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2db6t" event={"ID":"69aed169-e8af-429d-99c3-3b20be64e351","Type":"ContainerDied","Data":"dddf4eda2f0cba7e6db049d69dae8f5fca0c25f385573de83fc4a6a76dad2af5"}
Jul 16 11:03:52.499987 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:52.499967    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2db6t" event={"ID":"69aed169-e8af-429d-99c3-3b20be64e351","Type":"ContainerStarted","Data":"ae25e6f71d8cda02a8ee6c2bf5a84e9b88fdd3a35a8497c80babf4b6272fbd94"}
Jul 16 11:03:52.501310 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:52.501285    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-qxpqt" event={"ID":"c24aaddd-b44c-4155-a7c5-fadb281f9b39","Type":"ContainerStarted","Data":"0446b602a6cc36b232d8c774bdc968da26529236fa1fa486767ddf4a7dc2a991"}
Jul 16 11:03:52.502798 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:52.502775    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-zs9vs" event={"ID":"5baff3f6-85da-46fb-b42b-2e70e7d77cc4","Type":"ContainerStarted","Data":"43f16886ceeb83ab9140880d1f3ac03bdddd2e94dbe8e43aaeab7ef714a19b6b"}
Jul 16 11:03:52.504275 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:52.504257    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-z5dtv" event={"ID":"a3d44785-92d8-4a5e-bc2e-b91f4fa8b509","Type":"ContainerStarted","Data":"82a084adb92b4ba34a88fdd7fe85389c2c144c7d4ee14175fac52be0ccaea1ff"}
Jul 16 11:03:52.505854 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:52.505834    2583 generic.go:356] "Generic (PLEG): container finished" podID="6fd1e802-78a8-4991-99ee-47cfea84ab62" containerID="b25d724d7954ea65cc7c7de7af74bd33c1a3b44a90707bf929181aef48ffb6ea" exitCode=0
Jul 16 11:03:52.505943 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:52.505895    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8r89z" event={"ID":"6fd1e802-78a8-4991-99ee-47cfea84ab62","Type":"ContainerDied","Data":"b25d724d7954ea65cc7c7de7af74bd33c1a3b44a90707bf929181aef48ffb6ea"}
Jul 16 11:03:52.507629 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:52.507574    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-cr2x7" event={"ID":"8de3fcaf-5b76-4d0f-8453-60b892832242","Type":"ContainerStarted","Data":"5416375afb71a17e4053b62a2dcb96a143b6ece8072f321ef696c60d897178ab"}
Jul 16 11:03:52.510398 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:52.510331    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-28.ec2.internal" podStartSLOduration=22.510316185 podStartE2EDuration="22.510316185s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 11:03:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 11:03:33.46161072 +0000 UTC m=+6.095832178" watchObservedRunningTime="2026-07-16 11:03:52.510316185 +0000 UTC m=+25.144537656"
Jul 16 11:03:52.510584 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:52.510557    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-28.ec2.internal" podStartSLOduration=22.510549978 podStartE2EDuration="22.510549978s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 11:03:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 11:03:52.509954214 +0000 UTC m=+25.144175668" watchObservedRunningTime="2026-07-16 11:03:52.510549978 +0000 UTC m=+25.144771432"
Jul 16 11:03:52.525786 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:52.525739    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-qxpqt" podStartSLOduration=2.415983121 podStartE2EDuration="22.525713945s" totalImagesPullingTime="20.109730824s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 11:03:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 11:03:52.525214216 +0000 UTC m=+25.159435672" watchObservedRunningTime="2026-07-16 11:03:52.525713945 +0000 UTC m=+25.159935401"
Jul 16 11:03:52.559158 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:52.559100    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-zs9vs" podStartSLOduration=3.401331923 podStartE2EDuration="23.559080631s" totalImagesPullingTime="20.157748708s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 11:03:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 11:03:52.558837382 +0000 UTC m=+25.193058830" watchObservedRunningTime="2026-07-16 11:03:52.559080631 +0000 UTC m=+25.193302086"
Jul 16 11:03:52.559538 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:52.559507    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-z5dtv" podStartSLOduration=2.459147455 podStartE2EDuration="22.559497269s" totalImagesPullingTime="20.100349814s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 11:03:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 11:03:52.5410173 +0000 UTC m=+25.175238754" watchObservedRunningTime="2026-07-16 11:03:52.559497269 +0000 UTC m=+25.193718723"
Jul 16 11:03:52.593574 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:52.593518    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-cr2x7" podStartSLOduration=2.480756775 podStartE2EDuration="22.593501742s" totalImagesPullingTime="20.112744967s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 11:03:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 11:03:52.577302962 +0000 UTC m=+25.211524437" watchObservedRunningTime="2026-07-16 11:03:52.593501742 +0000 UTC m=+25.227723196"
Jul 16 11:03:52.620832 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:52.620784    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-6xfs6" podStartSLOduration=2.526256643 podStartE2EDuration="22.620767384s" totalImagesPullingTime="20.094510741s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 11:03:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 11:03:52.593193887 +0000 UTC m=+25.227415340" watchObservedRunningTime="2026-07-16 11:03:52.620767384 +0000 UTC m=+25.254988837"
Jul 16 11:03:53.105566 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:53.105524    2583 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 16 11:03:53.161416 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:53.161305    2583 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-16T11:03:53.105544013Z","UUID":"c20fd032-9807-4412-9f8a-7789d75de1fb","Handler":null,"Name":"","Endpoint":""}
Jul 16 11:03:53.162918 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:53.162895    2583 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 16 11:03:53.163049 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:53.162939    2583 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 16 11:03:53.223464 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:53.223419    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qcl5d"
Jul 16 11:03:53.223689 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:03:53.223566    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qcl5d" podUID="d4e7c597-9836-4a29-8591-e56938995710"
Jul 16 11:03:53.223689 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:53.223622    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rc7w4"
Jul 16 11:03:53.223800 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:03:53.223731    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rc7w4" podUID="3d0ee8e8-4018-4963-b553-af9b6ecb0f80"
Jul 16 11:03:53.512404 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:53.512367    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wtqvh" event={"ID":"21e8d664-4b0d-45cb-914f-7f55bb26d1f0","Type":"ContainerStarted","Data":"868a2c7f44debb4d2e6f49108f069680529bfe1b8e0cfa5e032b268954d63eea"}
Jul 16 11:03:53.513970 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:53.513916    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-x6lfk" event={"ID":"2d4103af-f06a-4592-83e0-8625e6267267","Type":"ContainerStarted","Data":"760661abb32d85affc8cee79065c326ce01ebdc7b2f2509363d7ac5fecccb4db"}
Jul 16 11:03:53.539028 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:53.538973    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-x6lfk" podStartSLOduration=3.444944433 podStartE2EDuration="23.538955026s" totalImagesPullingTime="20.094010593s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 11:03:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 11:03:53.538818121 +0000 UTC m=+26.173039574" watchObservedRunningTime="2026-07-16 11:03:53.538955026 +0000 UTC m=+26.173176480"
Jul 16 11:03:54.223383 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:54.223347    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hlm2s"
Jul 16 11:03:54.223695 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:03:54.223462    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hlm2s" podUID="c88b6ce0-4926-4de9-942e-7f1b30c33e9e"
Jul 16 11:03:55.223131 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:55.223094    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qcl5d"
Jul 16 11:03:55.223574 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:03:55.223242    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qcl5d" podUID="d4e7c597-9836-4a29-8591-e56938995710"
Jul 16 11:03:55.223574 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:55.223288    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rc7w4"
Jul 16 11:03:55.223574 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:03:55.223378    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rc7w4" podUID="3d0ee8e8-4018-4963-b553-af9b6ecb0f80"
Jul 16 11:03:55.520701 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:55.520591    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wtqvh" event={"ID":"21e8d664-4b0d-45cb-914f-7f55bb26d1f0","Type":"ContainerStarted","Data":"94ab96b08e69b02b31e63fbc0d62b667bf540d05697c6bc29cf51777bf37bc10"}
Jul 16 11:03:55.524304 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:55.524276    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2db6t_69aed169-e8af-429d-99c3-3b20be64e351/ovn-acl-logging/0.log"
Jul 16 11:03:55.524712 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:55.524690    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2db6t" event={"ID":"69aed169-e8af-429d-99c3-3b20be64e351","Type":"ContainerStarted","Data":"ffa4197cd1b3472599da349603123dbe28772ad27a772b5b9dcef151a3ec7a07"}
Jul 16 11:03:55.539673 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:55.539613    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wtqvh" podStartSLOduration=3.842297031 podStartE2EDuration="26.539579708s" totalImagesPullingTime="22.697282677s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 11:03:29 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 11:03:55.539507503 +0000 UTC m=+28.173728971" watchObservedRunningTime="2026-07-16 11:03:55.539579708 +0000 UTC m=+28.173801161"
Jul 16 11:03:56.223270 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:56.223093    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hlm2s"
Jul 16 11:03:56.223687 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:03:56.223343    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hlm2s" podUID="c88b6ce0-4926-4de9-942e-7f1b30c33e9e"
Jul 16 11:03:57.041567 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:57.041320    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-qxpqt"
Jul 16 11:03:57.041994 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:57.041971    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-qxpqt"
Jul 16 11:03:57.222677 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:57.222553    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qcl5d"
Jul 16 11:03:57.222813 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:03:57.222688    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qcl5d" podUID="d4e7c597-9836-4a29-8591-e56938995710"
Jul 16 11:03:57.222813 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:57.222745    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rc7w4"
Jul 16 11:03:57.222894 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:03:57.222830    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rc7w4" podUID="3d0ee8e8-4018-4963-b553-af9b6ecb0f80"
Jul 16 11:03:57.531713 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:57.531622    2583 generic.go:356] "Generic (PLEG): container finished" podID="6fd1e802-78a8-4991-99ee-47cfea84ab62" containerID="6e9d4bb00ac032c0db89b714641e04d99b4edda8ae72b339110db1f1d7aa7e68" exitCode=0
Jul 16 11:03:57.532631 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:57.531717    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8r89z" event={"ID":"6fd1e802-78a8-4991-99ee-47cfea84ab62","Type":"ContainerDied","Data":"6e9d4bb00ac032c0db89b714641e04d99b4edda8ae72b339110db1f1d7aa7e68"}
Jul 16 11:03:57.535248 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:57.535210    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2db6t_69aed169-e8af-429d-99c3-3b20be64e351/ovn-acl-logging/0.log"
Jul 16 11:03:57.535639 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:57.535615    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2db6t" event={"ID":"69aed169-e8af-429d-99c3-3b20be64e351","Type":"ContainerStarted","Data":"2fd47c5624df638a44012f4b0ca5dd8214e81a8daec65bf21f33c02c3146e91d"}
Jul 16 11:03:57.535817 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:57.535801    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-qxpqt"
Jul 16 11:03:57.536035 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:57.536015    2583 scope.go:117] "RemoveContainer" containerID="dddf4eda2f0cba7e6db049d69dae8f5fca0c25f385573de83fc4a6a76dad2af5"
Jul 16 11:03:57.536210 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:57.536197    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-qxpqt"
Jul 16 11:03:58.222607 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:58.222573    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hlm2s"
Jul 16 11:03:58.222845 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:03:58.222710    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hlm2s" podUID="c88b6ce0-4926-4de9-942e-7f1b30c33e9e"
Jul 16 11:03:58.541918 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:58.541840    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2db6t_69aed169-e8af-429d-99c3-3b20be64e351/ovn-acl-logging/0.log"
Jul 16 11:03:58.542389 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:58.542251    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2db6t" event={"ID":"69aed169-e8af-429d-99c3-3b20be64e351","Type":"ContainerStarted","Data":"76bf16902d4911ee750021bf822c33f1334d5b3e6a87e4048babd8dc78164e90"}
Jul 16 11:03:58.542389 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:58.542360    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-2db6t"
Jul 16 11:03:58.542513 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:58.542470    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-2db6t"
Jul 16 11:03:58.542738 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:58.542716    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-2db6t"
Jul 16 11:03:58.558063 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:58.558038    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-2db6t"
Jul 16 11:03:58.558230 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:58.558173    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-2db6t"
Jul 16 11:03:58.579769 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:58.579722    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-2db6t" podStartSLOduration=8.405509054 podStartE2EDuration="28.579705571s" totalImagesPullingTime="20.174196517s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 11:03:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 11:03:58.57928933 +0000 UTC m=+31.213510804" watchObservedRunningTime="2026-07-16 11:03:58.579705571 +0000 UTC m=+31.213927024"
Jul 16 11:03:59.221365 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:59.221288    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-hlm2s"]
Jul 16 11:03:59.221570 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:59.221423    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hlm2s"
Jul 16 11:03:59.221570 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:03:59.221502    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hlm2s" podUID="c88b6ce0-4926-4de9-942e-7f1b30c33e9e"
Jul 16 11:03:59.223888 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:59.223860    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qcl5d"
Jul 16 11:03:59.223999 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:03:59.223975    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qcl5d" podUID="d4e7c597-9836-4a29-8591-e56938995710"
Jul 16 11:03:59.224043 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:59.224013    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rc7w4"
Jul 16 11:03:59.224101 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:03:59.224083    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rc7w4" podUID="3d0ee8e8-4018-4963-b553-af9b6ecb0f80"
Jul 16 11:03:59.226558 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:59.226539    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-rc7w4"]
Jul 16 11:03:59.226669 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:59.226578    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-qcl5d"]
Jul 16 11:03:59.546551 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:59.546460    2583 generic.go:356] "Generic (PLEG): container finished" podID="6fd1e802-78a8-4991-99ee-47cfea84ab62" containerID="a93d55bc8110184510b24b65e931126907a0f14536055424199de7fe6429f2e3" exitCode=0
Jul 16 11:03:59.547018 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:59.546556    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8r89z" event={"ID":"6fd1e802-78a8-4991-99ee-47cfea84ab62","Type":"ContainerDied","Data":"a93d55bc8110184510b24b65e931126907a0f14536055424199de7fe6429f2e3"}
Jul 16 11:03:59.547018 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:59.546607    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qcl5d"
Jul 16 11:03:59.547018 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:03:59.546704    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rc7w4"
Jul 16 11:03:59.547018 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:03:59.546716    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qcl5d" podUID="d4e7c597-9836-4a29-8591-e56938995710"
Jul 16 11:03:59.547018 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:03:59.546807    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rc7w4" podUID="3d0ee8e8-4018-4963-b553-af9b6ecb0f80"
Jul 16 11:04:01.223231 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:01.223202    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hlm2s"
Jul 16 11:04:01.223552 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:01.223199    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rc7w4"
Jul 16 11:04:01.223552 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:04:01.223323    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hlm2s" podUID="c88b6ce0-4926-4de9-942e-7f1b30c33e9e"
Jul 16 11:04:01.223552 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:04:01.223413    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rc7w4" podUID="3d0ee8e8-4018-4963-b553-af9b6ecb0f80"
Jul 16 11:04:01.223552 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:01.223199    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qcl5d"
Jul 16 11:04:01.223552 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:04:01.223496    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qcl5d" podUID="d4e7c597-9836-4a29-8591-e56938995710"
Jul 16 11:04:01.554379 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:01.554349    2583 generic.go:356] "Generic (PLEG): container finished" podID="6fd1e802-78a8-4991-99ee-47cfea84ab62" containerID="ab56aa7925b3e9a8a32f80e538e4848bef3b079e87d38eba40d2fe7384992278" exitCode=0
Jul 16 11:04:01.554554 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:01.554423    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8r89z" event={"ID":"6fd1e802-78a8-4991-99ee-47cfea84ab62","Type":"ContainerDied","Data":"ab56aa7925b3e9a8a32f80e538e4848bef3b079e87d38eba40d2fe7384992278"}
Jul 16 11:04:02.851354 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:02.851319    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d4e7c597-9836-4a29-8591-e56938995710-metrics-certs\") pod \"network-metrics-daemon-qcl5d\" (UID: \"d4e7c597-9836-4a29-8591-e56938995710\") " pod="openshift-multus/network-metrics-daemon-qcl5d"
Jul 16 11:04:02.851746 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:04:02.851471    2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 11:04:02.851746 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:04:02.851536    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d4e7c597-9836-4a29-8591-e56938995710-metrics-certs podName:d4e7c597-9836-4a29-8591-e56938995710 nodeName:}" failed. No retries permitted until 2026-07-16 11:04:34.851520136 +0000 UTC m=+67.485741567 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d4e7c597-9836-4a29-8591-e56938995710-metrics-certs") pod "network-metrics-daemon-qcl5d" (UID: "d4e7c597-9836-4a29-8591-e56938995710") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 11:04:02.952091 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:02.951882    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c88b6ce0-4926-4de9-942e-7f1b30c33e9e-original-pull-secret\") pod \"global-pull-secret-syncer-hlm2s\" (UID: \"c88b6ce0-4926-4de9-942e-7f1b30c33e9e\") " pod="kube-system/global-pull-secret-syncer-hlm2s"
Jul 16 11:04:02.952264 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:04:02.952035    2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 16 11:04:02.952264 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:04:02.952196    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c88b6ce0-4926-4de9-942e-7f1b30c33e9e-original-pull-secret podName:c88b6ce0-4926-4de9-942e-7f1b30c33e9e nodeName:}" failed. No retries permitted until 2026-07-16 11:04:34.952178094 +0000 UTC m=+67.586399529 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c88b6ce0-4926-4de9-942e-7f1b30c33e9e-original-pull-secret") pod "global-pull-secret-syncer-hlm2s" (UID: "c88b6ce0-4926-4de9-942e-7f1b30c33e9e") : object "kube-system"/"original-pull-secret" not registered
Jul 16 11:04:03.053139 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:03.053106    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2kcc2\" (UniqueName: \"kubernetes.io/projected/3d0ee8e8-4018-4963-b553-af9b6ecb0f80-kube-api-access-2kcc2\") pod \"network-check-target-rc7w4\" (UID: \"3d0ee8e8-4018-4963-b553-af9b6ecb0f80\") " pod="openshift-network-diagnostics/network-check-target-rc7w4"
Jul 16 11:04:03.053291 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:04:03.053275    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 16 11:04:03.053346 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:04:03.053298    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 16 11:04:03.053346 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:04:03.053308    2583 projected.go:196] Error preparing data for projected volume kube-api-access-2kcc2 for pod openshift-network-diagnostics/network-check-target-rc7w4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 11:04:03.053424 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:04:03.053369    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3d0ee8e8-4018-4963-b553-af9b6ecb0f80-kube-api-access-2kcc2 podName:3d0ee8e8-4018-4963-b553-af9b6ecb0f80 nodeName:}" failed. No retries permitted until 2026-07-16 11:04:35.053349578 +0000 UTC m=+67.687571032 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-2kcc2" (UniqueName: "kubernetes.io/projected/3d0ee8e8-4018-4963-b553-af9b6ecb0f80-kube-api-access-2kcc2") pod "network-check-target-rc7w4" (UID: "3d0ee8e8-4018-4963-b553-af9b6ecb0f80") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 11:04:03.222851 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:03.222702    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qcl5d"
Jul 16 11:04:03.222851 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:03.222743    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hlm2s"
Jul 16 11:04:03.222851 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:03.222701    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rc7w4"
Jul 16 11:04:03.223128 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:04:03.222851    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qcl5d" podUID="d4e7c597-9836-4a29-8591-e56938995710"
Jul 16 11:04:03.223128 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:04:03.222939    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rc7w4" podUID="3d0ee8e8-4018-4963-b553-af9b6ecb0f80"
Jul 16 11:04:03.223128 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:04:03.223009    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hlm2s" podUID="c88b6ce0-4926-4de9-942e-7f1b30c33e9e"
Jul 16 11:04:05.222747 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.222712    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qcl5d"
Jul 16 11:04:05.222747 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.222748    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rc7w4"
Jul 16 11:04:05.223233 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.222711    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hlm2s"
Jul 16 11:04:05.223233 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:04:05.222856    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qcl5d" podUID="d4e7c597-9836-4a29-8591-e56938995710"
Jul 16 11:04:05.223233 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:04:05.222948    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hlm2s" podUID="c88b6ce0-4926-4de9-942e-7f1b30c33e9e"
Jul 16 11:04:05.223233 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:04:05.223044    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rc7w4" podUID="3d0ee8e8-4018-4963-b553-af9b6ecb0f80"
Jul 16 11:04:05.671008 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.670978    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-28.ec2.internal" event="NodeReady"
Jul 16 11:04:05.671197 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.671145    2583 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 16 11:04:05.707118 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.707085    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-d496q"]
Jul 16 11:04:05.710181 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.710143    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-d496q"
Jul 16 11:04:05.710571 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.710519    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-65f68df599-zw6jf"]
Jul 16 11:04:05.712581 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.712563    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-9ztqs"]
Jul 16 11:04:05.712726 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.712706    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-65f68df599-zw6jf"
Jul 16 11:04:05.712938 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.712912    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Jul 16 11:04:05.713037 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.712989    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-b7rfj\""
Jul 16 11:04:05.713257 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.713146    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Jul 16 11:04:05.714345 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.714325    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-76cb9bc7db-x2cnj"]
Jul 16 11:04:05.714467 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.714448    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-9ztqs"
Jul 16 11:04:05.715113 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.715093    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 16 11:04:05.715209 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.715160    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-2nzj5\""
Jul 16 11:04:05.715481 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.715463    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 16 11:04:05.716286 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.716249    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 16 11:04:05.716588 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.716568    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-lx49b"]
Jul 16 11:04:05.716719 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.716705    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-76cb9bc7db-x2cnj"
Jul 16 11:04:05.718319 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.718302    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-85d88"]
Jul 16 11:04:05.719100 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.718372    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-68d9dd94cf-lx49b"
Jul 16 11:04:05.720146 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.720124    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-85d88"
Jul 16 11:04:05.721225 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.721205    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-jfl46\""
Jul 16 11:04:05.721675 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.721418    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Jul 16 11:04:05.721799 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.721743    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 16 11:04:05.721799 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.721753    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 16 11:04:05.721899 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.721853    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Jul 16 11:04:05.721996 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.721976    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 16 11:04:05.722785 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.722765    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 16 11:04:05.722895 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.722793    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Jul 16 11:04:05.722895 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.722877    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-d496q"]
Jul 16 11:04:05.723038 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.723022    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Jul 16 11:04:05.723103 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.723089    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Jul 16 11:04:05.723217 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.723201    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Jul 16 11:04:05.723287 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.723096    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Jul 16 11:04:05.723462 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.723445    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 16 11:04:05.723860 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.723830    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-jsj9w\""
Jul 16 11:04:05.724108 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.724092    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Jul 16 11:04:05.724357 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.724338    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 16 11:04:05.725098 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.725080    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-sdh6l\""
Jul 16 11:04:05.727298 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.727271    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Jul 16 11:04:05.727581 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.727560    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Jul 16 11:04:05.730422 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.730402    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Jul 16 11:04:05.730516 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.730405    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-dvvf6\""
Jul 16 11:04:05.730837 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.730407    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Jul 16 11:04:05.730837 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.730764    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 16 11:04:05.731222 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.731198    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-5649t"]
Jul 16 11:04:05.733750 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.733733    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-5649t"
Jul 16 11:04:05.733899 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.733871    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Jul 16 11:04:05.734281 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.734258    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-982zd"]
Jul 16 11:04:05.737294 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.737263    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-7866497494-9lm5s"]
Jul 16 11:04:05.737739 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.737566    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-982zd"
Jul 16 11:04:05.738813 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.738789    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 16 11:04:05.738813 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.738805    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 16 11:04:05.739350 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.739257    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-kmdfj\""
Jul 16 11:04:05.740629 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.740489    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-7d8g4"]
Jul 16 11:04:05.741025 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.741007    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7866497494-9lm5s"
Jul 16 11:04:05.741406 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.741385    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Jul 16 11:04:05.741628 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.741611    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Jul 16 11:04:05.741797 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.741779    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-7mjv2\""
Jul 16 11:04:05.742017 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.741998    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Jul 16 11:04:05.742559 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.742539    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-7gl88"]
Jul 16 11:04:05.742716 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.742699    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-7d8g4"
Jul 16 11:04:05.743453 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.743437    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Jul 16 11:04:05.743719 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.743672    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-zkvqv\""
Jul 16 11:04:05.744004 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.743988    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Jul 16 11:04:05.744179 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.744050    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Jul 16 11:04:05.744743 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.744469    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-4mhsn"]
Jul 16 11:04:05.744743 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.744573    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Jul 16 11:04:05.745082 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.745051    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 16 11:04:05.745192 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.745177    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-7gl88"
Jul 16 11:04:05.746003 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.745987    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-lqhs2\""
Jul 16 11:04:05.746401 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.746386    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-9ztqs"]
Jul 16 11:04:05.746514 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.746496    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-4mhsn"
Jul 16 11:04:05.746619 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.746411    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 16 11:04:05.746840 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.746505    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-85d88"]
Jul 16 11:04:05.746975 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.746958    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-65f68df599-zw6jf"]
Jul 16 11:04:05.747048 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.746987    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-76cb9bc7db-x2cnj"]
Jul 16 11:04:05.748228 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.748203    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Jul 16 11:04:05.748405 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.748386    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Jul 16 11:04:05.748480 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.748441    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Jul 16 11:04:05.748935 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.748917    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-ssb7j\""
Jul 16 11:04:05.749202 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.749169    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Jul 16 11:04:05.750932 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.750914    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 16 11:04:05.751163 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.751145    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-45jx4\""
Jul 16 11:04:05.751380 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.751360    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 16 11:04:05.751510 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.751232    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 16 11:04:05.752853 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.752838    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-7d8g4"]
Jul 16 11:04:05.754105 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.754085    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Jul 16 11:04:05.754403 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.754388    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7866497494-9lm5s"]
Jul 16 11:04:05.756573 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.756548    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-4mhsn"]
Jul 16 11:04:05.761932 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.758564    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-lx49b"]
Jul 16 11:04:05.763259 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.762108    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-5649t"]
Jul 16 11:04:05.763259 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.763142    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-982zd"]
Jul 16 11:04:05.767639 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.767618    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-7gl88"]
Jul 16 11:04:05.874085 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.874053    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ec3eb24d-8f3c-40b7-bfe9-a34fccdc20cd-service-ca-bundle\") pod \"router-default-76cb9bc7db-x2cnj\" (UID: \"ec3eb24d-8f3c-40b7-bfe9-a34fccdc20cd\") " pod="openshift-ingress/router-default-76cb9bc7db-x2cnj"
Jul 16 11:04:05.874085 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.874086    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/543513a9-883b-4f90-8478-eb51b3186af1-serving-cert\") pod \"service-ca-operator-7695fc6c8c-7gl88\" (UID: \"543513a9-883b-4f90-8478-eb51b3186af1\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-7gl88"
Jul 16 11:04:05.874327 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.874102    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6a4b5dde-f664-4d97-889e-bfe4d4df1416-registry-tls\") pod \"image-registry-65f68df599-zw6jf\" (UID: \"6a4b5dde-f664-4d97-889e-bfe4d4df1416\") " pod="openshift-image-registry/image-registry-65f68df599-zw6jf"
Jul 16 11:04:05.874327 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.874128    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f7hst\" (UniqueName: \"kubernetes.io/projected/369fdbca-23e1-4389-9b69-4f09463fc7bc-kube-api-access-f7hst\") pod \"network-check-source-869dcfcdcd-7d8g4\" (UID: \"369fdbca-23e1-4389-9b69-4f09463fc7bc\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-7d8g4"
Jul 16 11:04:05.874327 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.874163    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/921e629b-d5a3-4bf6-8bad-0131d0a812ea-config-volume\") pod \"dns-default-5649t\" (UID: \"921e629b-d5a3-4bf6-8bad-0131d0a812ea\") " pod="openshift-dns/dns-default-5649t"
Jul 16 11:04:05.874327 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.874218    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c839c027-e922-48f5-9940-d293b38aef05-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-lx49b\" (UID: \"c839c027-e922-48f5-9940-d293b38aef05\") " pod="openshift-insights/insights-operator-68d9dd94cf-lx49b"
Jul 16 11:04:05.874327 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.874246    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ec3eb24d-8f3c-40b7-bfe9-a34fccdc20cd-metrics-certs\") pod \"router-default-76cb9bc7db-x2cnj\" (UID: \"ec3eb24d-8f3c-40b7-bfe9-a34fccdc20cd\") " pod="openshift-ingress/router-default-76cb9bc7db-x2cnj"
Jul 16 11:04:05.874327 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.874270    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c839c027-e922-48f5-9940-d293b38aef05-tmp\") pod \"insights-operator-68d9dd94cf-lx49b\" (UID: \"c839c027-e922-48f5-9940-d293b38aef05\") " pod="openshift-insights/insights-operator-68d9dd94cf-lx49b"
Jul 16 11:04:05.874327 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.874301    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2mxl4\" (UniqueName: \"kubernetes.io/projected/6a4b5dde-f664-4d97-889e-bfe4d4df1416-kube-api-access-2mxl4\") pod \"image-registry-65f68df599-zw6jf\" (UID: \"6a4b5dde-f664-4d97-889e-bfe4d4df1416\") " pod="openshift-image-registry/image-registry-65f68df599-zw6jf"
Jul 16 11:04:05.874694 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.874356    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/ec3eb24d-8f3c-40b7-bfe9-a34fccdc20cd-default-certificate\") pod \"router-default-76cb9bc7db-x2cnj\" (UID: \"ec3eb24d-8f3c-40b7-bfe9-a34fccdc20cd\") " pod="openshift-ingress/router-default-76cb9bc7db-x2cnj"
Jul 16 11:04:05.874694 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.874391    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8dcffe9f-afd4-4e68-9c2a-a6d824df311e-cert\") pod \"ingress-canary-4mhsn\" (UID: \"8dcffe9f-afd4-4e68-9c2a-a6d824df311e\") " pod="openshift-ingress-canary/ingress-canary-4mhsn"
Jul 16 11:04:05.874694 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.874416    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/2ae4a146-99bf-422f-982f-0dab385b060b-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-9ztqs\" (UID: \"2ae4a146-99bf-422f-982f-0dab385b060b\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-9ztqs"
Jul 16 11:04:05.874694 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.874440    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f5zqf\" (UniqueName: \"kubernetes.io/projected/f64f15a3-5a9f-4737-bc4f-73d5ec311e41-kube-api-access-f5zqf\") pod \"console-operator-7866497494-9lm5s\" (UID: \"f64f15a3-5a9f-4737-bc4f-73d5ec311e41\") " pod="openshift-console-operator/console-operator-7866497494-9lm5s"
Jul 16 11:04:05.874694 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.874490    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/628849b1-f592-48a1-af55-d6f78351be63-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-982zd\" (UID: \"628849b1-f592-48a1-af55-d6f78351be63\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-982zd"
Jul 16 11:04:05.874694 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.874519    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/6a4b5dde-f664-4d97-889e-bfe4d4df1416-ca-trust-extracted\") pod \"image-registry-65f68df599-zw6jf\" (UID: \"6a4b5dde-f664-4d97-889e-bfe4d4df1416\") " pod="openshift-image-registry/image-registry-65f68df599-zw6jf"
Jul 16 11:04:05.874694 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.874553    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-44gnb\" (UniqueName: \"kubernetes.io/projected/c839c027-e922-48f5-9940-d293b38aef05-kube-api-access-44gnb\") pod \"insights-operator-68d9dd94cf-lx49b\" (UID: \"c839c027-e922-48f5-9940-d293b38aef05\") " pod="openshift-insights/insights-operator-68d9dd94cf-lx49b"
Jul 16 11:04:05.874694 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.874584    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vgps7\" (UniqueName: \"kubernetes.io/projected/ca7a1d19-213d-4ce9-8a34-4a93ddcab230-kube-api-access-vgps7\") pod \"volume-data-source-validator-78495597f6-d496q\" (UID: \"ca7a1d19-213d-4ce9-8a34-4a93ddcab230\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-d496q"
Jul 16 11:04:05.874694 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.874635    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sc6xs\" (UniqueName: \"kubernetes.io/projected/8dcffe9f-afd4-4e68-9c2a-a6d824df311e-kube-api-access-sc6xs\") pod \"ingress-canary-4mhsn\" (UID: \"8dcffe9f-afd4-4e68-9c2a-a6d824df311e\") " pod="openshift-ingress-canary/ingress-canary-4mhsn"
Jul 16 11:04:05.874694 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.874659    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/921e629b-d5a3-4bf6-8bad-0131d0a812ea-tmp-dir\") pod \"dns-default-5649t\" (UID: \"921e629b-d5a3-4bf6-8bad-0131d0a812ea\") " pod="openshift-dns/dns-default-5649t"
Jul 16 11:04:05.875058 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.874713    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8tgg7\" (UniqueName: \"kubernetes.io/projected/628849b1-f592-48a1-af55-d6f78351be63-kube-api-access-8tgg7\") pod \"cluster-samples-operator-65db858894-982zd\" (UID: \"628849b1-f592-48a1-af55-d6f78351be63\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-982zd"
Jul 16 11:04:05.875058 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.874776    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6a4b5dde-f664-4d97-889e-bfe4d4df1416-bound-sa-token\") pod \"image-registry-65f68df599-zw6jf\" (UID: \"6a4b5dde-f664-4d97-889e-bfe4d4df1416\") " pod="openshift-image-registry/image-registry-65f68df599-zw6jf"
Jul 16 11:04:05.875058 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.874811    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2ae4a146-99bf-422f-982f-0dab385b060b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-9ztqs\" (UID: \"2ae4a146-99bf-422f-982f-0dab385b060b\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-9ztqs"
Jul 16 11:04:05.875058 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.874903    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f64f15a3-5a9f-4737-bc4f-73d5ec311e41-trusted-ca\") pod \"console-operator-7866497494-9lm5s\" (UID: \"f64f15a3-5a9f-4737-bc4f-73d5ec311e41\") " pod="openshift-console-operator/console-operator-7866497494-9lm5s"
Jul 16 11:04:05.875058 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.874928    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c839c027-e922-48f5-9940-d293b38aef05-serving-cert\") pod \"insights-operator-68d9dd94cf-lx49b\" (UID: \"c839c027-e922-48f5-9940-d293b38aef05\") " pod="openshift-insights/insights-operator-68d9dd94cf-lx49b"
Jul 16 11:04:05.875058 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.874951    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8n8zc\" (UniqueName: \"kubernetes.io/projected/921e629b-d5a3-4bf6-8bad-0131d0a812ea-kube-api-access-8n8zc\") pod \"dns-default-5649t\" (UID: \"921e629b-d5a3-4bf6-8bad-0131d0a812ea\") " pod="openshift-dns/dns-default-5649t"
Jul 16 11:04:05.875058 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.874970    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/543513a9-883b-4f90-8478-eb51b3186af1-config\") pod \"service-ca-operator-7695fc6c8c-7gl88\" (UID: \"543513a9-883b-4f90-8478-eb51b3186af1\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-7gl88"
Jul 16 11:04:05.875058 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.874997    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n9f56\" (UniqueName: \"kubernetes.io/projected/e1f38c54-19be-4f6f-872b-39093678a7ad-kube-api-access-n9f56\") pod \"kube-storage-version-migrator-operator-799cb7f958-85d88\" (UID: \"e1f38c54-19be-4f6f-872b-39093678a7ad\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-85d88"
Jul 16 11:04:05.875058 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.875030    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/6a4b5dde-f664-4d97-889e-bfe4d4df1416-image-registry-private-configuration\") pod \"image-registry-65f68df599-zw6jf\" (UID: \"6a4b5dde-f664-4d97-889e-bfe4d4df1416\") " pod="openshift-image-registry/image-registry-65f68df599-zw6jf"
Jul 16 11:04:05.875058 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.875058    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f64f15a3-5a9f-4737-bc4f-73d5ec311e41-config\") pod \"console-operator-7866497494-9lm5s\" (UID: \"f64f15a3-5a9f-4737-bc4f-73d5ec311e41\") " pod="openshift-console-operator/console-operator-7866497494-9lm5s"
Jul 16 11:04:05.875481 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.875163    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/c839c027-e922-48f5-9940-d293b38aef05-snapshots\") pod \"insights-operator-68d9dd94cf-lx49b\" (UID: \"c839c027-e922-48f5-9940-d293b38aef05\") " pod="openshift-insights/insights-operator-68d9dd94cf-lx49b"
Jul 16 11:04:05.875481 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.875193    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e1f38c54-19be-4f6f-872b-39093678a7ad-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-85d88\" (UID: \"e1f38c54-19be-4f6f-872b-39093678a7ad\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-85d88"
Jul 16 11:04:05.875481 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.875217    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sb2sl\" (UniqueName: \"kubernetes.io/projected/543513a9-883b-4f90-8478-eb51b3186af1-kube-api-access-sb2sl\") pod \"service-ca-operator-7695fc6c8c-7gl88\" (UID: \"543513a9-883b-4f90-8478-eb51b3186af1\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-7gl88"
Jul 16 11:04:05.875481 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.875250    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f64f15a3-5a9f-4737-bc4f-73d5ec311e41-serving-cert\") pod \"console-operator-7866497494-9lm5s\" (UID: \"f64f15a3-5a9f-4737-bc4f-73d5ec311e41\") " pod="openshift-console-operator/console-operator-7866497494-9lm5s"
Jul 16 11:04:05.875481 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.875304    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/921e629b-d5a3-4bf6-8bad-0131d0a812ea-metrics-tls\") pod \"dns-default-5649t\" (UID: \"921e629b-d5a3-4bf6-8bad-0131d0a812ea\") " pod="openshift-dns/dns-default-5649t"
Jul 16 11:04:05.875481 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.875351    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/6a4b5dde-f664-4d97-889e-bfe4d4df1416-registry-certificates\") pod \"image-registry-65f68df599-zw6jf\" (UID: \"6a4b5dde-f664-4d97-889e-bfe4d4df1416\") " pod="openshift-image-registry/image-registry-65f68df599-zw6jf"
Jul 16 11:04:05.875481 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.875403    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rkn9z\" (UniqueName: \"kubernetes.io/projected/2ae4a146-99bf-422f-982f-0dab385b060b-kube-api-access-rkn9z\") pod \"cluster-monitoring-operator-5f7f494dd4-9ztqs\" (UID: \"2ae4a146-99bf-422f-982f-0dab385b060b\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-9ztqs"
Jul 16 11:04:05.875481 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.875458    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6a4b5dde-f664-4d97-889e-bfe4d4df1416-trusted-ca\") pod \"image-registry-65f68df599-zw6jf\" (UID: \"6a4b5dde-f664-4d97-889e-bfe4d4df1416\") " pod="openshift-image-registry/image-registry-65f68df599-zw6jf"
Jul 16 11:04:05.875858 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.875485    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/6a4b5dde-f664-4d97-889e-bfe4d4df1416-installation-pull-secrets\") pod \"image-registry-65f68df599-zw6jf\" (UID: \"6a4b5dde-f664-4d97-889e-bfe4d4df1416\") " pod="openshift-image-registry/image-registry-65f68df599-zw6jf"
Jul 16 11:04:05.875858 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.875527    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e1f38c54-19be-4f6f-872b-39093678a7ad-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-85d88\" (UID: \"e1f38c54-19be-4f6f-872b-39093678a7ad\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-85d88"
Jul 16 11:04:05.875858 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.875553    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c839c027-e922-48f5-9940-d293b38aef05-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-lx49b\" (UID: \"c839c027-e922-48f5-9940-d293b38aef05\") " pod="openshift-insights/insights-operator-68d9dd94cf-lx49b"
Jul 16 11:04:05.875858 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.875581    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6n4lk\" (UniqueName: \"kubernetes.io/projected/ec3eb24d-8f3c-40b7-bfe9-a34fccdc20cd-kube-api-access-6n4lk\") pod \"router-default-76cb9bc7db-x2cnj\" (UID: \"ec3eb24d-8f3c-40b7-bfe9-a34fccdc20cd\") " pod="openshift-ingress/router-default-76cb9bc7db-x2cnj"
Jul 16 11:04:05.875858 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.875687    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/ec3eb24d-8f3c-40b7-bfe9-a34fccdc20cd-stats-auth\") pod \"router-default-76cb9bc7db-x2cnj\" (UID: \"ec3eb24d-8f3c-40b7-bfe9-a34fccdc20cd\") " pod="openshift-ingress/router-default-76cb9bc7db-x2cnj"
Jul 16 11:04:05.914568 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.914534    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qxpqt_c24aaddd-b44c-4155-a7c5-fadb281f9b39/konnectivity-agent/0.log" limit=1024
Jul 16 11:04:05.976205 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.976117    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f7hst\" (UniqueName: \"kubernetes.io/projected/369fdbca-23e1-4389-9b69-4f09463fc7bc-kube-api-access-f7hst\") pod \"network-check-source-869dcfcdcd-7d8g4\" (UID: \"369fdbca-23e1-4389-9b69-4f09463fc7bc\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-7d8g4"
Jul 16 11:04:05.976205 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.976159    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/921e629b-d5a3-4bf6-8bad-0131d0a812ea-config-volume\") pod \"dns-default-5649t\" (UID: \"921e629b-d5a3-4bf6-8bad-0131d0a812ea\") " pod="openshift-dns/dns-default-5649t"
Jul 16 11:04:05.976205 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.976183    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c839c027-e922-48f5-9940-d293b38aef05-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-lx49b\" (UID: \"c839c027-e922-48f5-9940-d293b38aef05\") " pod="openshift-insights/insights-operator-68d9dd94cf-lx49b"
Jul 16 11:04:05.976471 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.976207    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ec3eb24d-8f3c-40b7-bfe9-a34fccdc20cd-metrics-certs\") pod \"router-default-76cb9bc7db-x2cnj\" (UID: \"ec3eb24d-8f3c-40b7-bfe9-a34fccdc20cd\") " pod="openshift-ingress/router-default-76cb9bc7db-x2cnj"
Jul 16 11:04:05.976471 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.976233    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c839c027-e922-48f5-9940-d293b38aef05-tmp\") pod \"insights-operator-68d9dd94cf-lx49b\" (UID: \"c839c027-e922-48f5-9940-d293b38aef05\") " pod="openshift-insights/insights-operator-68d9dd94cf-lx49b"
Jul 16 11:04:05.976471 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.976257    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2mxl4\" (UniqueName: \"kubernetes.io/projected/6a4b5dde-f664-4d97-889e-bfe4d4df1416-kube-api-access-2mxl4\") pod \"image-registry-65f68df599-zw6jf\" (UID: \"6a4b5dde-f664-4d97-889e-bfe4d4df1416\") " pod="openshift-image-registry/image-registry-65f68df599-zw6jf"
Jul 16 11:04:05.976471 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.976282    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/ec3eb24d-8f3c-40b7-bfe9-a34fccdc20cd-default-certificate\") pod \"router-default-76cb9bc7db-x2cnj\" (UID: \"ec3eb24d-8f3c-40b7-bfe9-a34fccdc20cd\") " pod="openshift-ingress/router-default-76cb9bc7db-x2cnj"
Jul 16 11:04:05.976471 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.976302    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8dcffe9f-afd4-4e68-9c2a-a6d824df311e-cert\") pod \"ingress-canary-4mhsn\" (UID: \"8dcffe9f-afd4-4e68-9c2a-a6d824df311e\") " pod="openshift-ingress-canary/ingress-canary-4mhsn"
Jul 16 11:04:05.976471 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.976323    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/2ae4a146-99bf-422f-982f-0dab385b060b-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-9ztqs\" (UID: \"2ae4a146-99bf-422f-982f-0dab385b060b\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-9ztqs"
Jul 16 11:04:05.976471 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.976345    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f5zqf\" (UniqueName: \"kubernetes.io/projected/f64f15a3-5a9f-4737-bc4f-73d5ec311e41-kube-api-access-f5zqf\") pod \"console-operator-7866497494-9lm5s\" (UID: \"f64f15a3-5a9f-4737-bc4f-73d5ec311e41\") " pod="openshift-console-operator/console-operator-7866497494-9lm5s"
Jul 16 11:04:05.976471 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.976378    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/628849b1-f592-48a1-af55-d6f78351be63-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-982zd\" (UID: \"628849b1-f592-48a1-af55-d6f78351be63\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-982zd"
Jul 16 11:04:05.976471 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.976406    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/6a4b5dde-f664-4d97-889e-bfe4d4df1416-ca-trust-extracted\") pod \"image-registry-65f68df599-zw6jf\" (UID: \"6a4b5dde-f664-4d97-889e-bfe4d4df1416\") " pod="openshift-image-registry/image-registry-65f68df599-zw6jf"
Jul 16 11:04:05.976471 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.976441    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-44gnb\" (UniqueName: \"kubernetes.io/projected/c839c027-e922-48f5-9940-d293b38aef05-kube-api-access-44gnb\") pod \"insights-operator-68d9dd94cf-lx49b\" (UID: \"c839c027-e922-48f5-9940-d293b38aef05\") " pod="openshift-insights/insights-operator-68d9dd94cf-lx49b"
Jul 16 11:04:05.976471 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.976471    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vgps7\" (UniqueName: \"kubernetes.io/projected/ca7a1d19-213d-4ce9-8a34-4a93ddcab230-kube-api-access-vgps7\") pod \"volume-data-source-validator-78495597f6-d496q\" (UID: \"ca7a1d19-213d-4ce9-8a34-4a93ddcab230\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-d496q"
Jul 16 11:04:05.977095 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.976529    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sc6xs\" (UniqueName: \"kubernetes.io/projected/8dcffe9f-afd4-4e68-9c2a-a6d824df311e-kube-api-access-sc6xs\") pod \"ingress-canary-4mhsn\" (UID: \"8dcffe9f-afd4-4e68-9c2a-a6d824df311e\") " pod="openshift-ingress-canary/ingress-canary-4mhsn"
Jul 16 11:04:05.977095 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.976574    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/921e629b-d5a3-4bf6-8bad-0131d0a812ea-tmp-dir\") pod \"dns-default-5649t\" (UID: \"921e629b-d5a3-4bf6-8bad-0131d0a812ea\") " pod="openshift-dns/dns-default-5649t"
Jul 16 11:04:05.977095 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:04:05.976582    2583 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 16 11:04:05.977095 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.976648    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8tgg7\" (UniqueName: \"kubernetes.io/projected/628849b1-f592-48a1-af55-d6f78351be63-kube-api-access-8tgg7\") pod \"cluster-samples-operator-65db858894-982zd\" (UID: \"628849b1-f592-48a1-af55-d6f78351be63\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-982zd"
Jul 16 11:04:05.977095 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:04:05.976682    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/628849b1-f592-48a1-af55-d6f78351be63-samples-operator-tls podName:628849b1-f592-48a1-af55-d6f78351be63 nodeName:}" failed. No retries permitted until 2026-07-16 11:04:06.476663598 +0000 UTC m=+39.110885035 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/628849b1-f592-48a1-af55-d6f78351be63-samples-operator-tls") pod "cluster-samples-operator-65db858894-982zd" (UID: "628849b1-f592-48a1-af55-d6f78351be63") : secret "samples-operator-tls" not found
Jul 16 11:04:05.977095 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.976711    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6a4b5dde-f664-4d97-889e-bfe4d4df1416-bound-sa-token\") pod \"image-registry-65f68df599-zw6jf\" (UID: \"6a4b5dde-f664-4d97-889e-bfe4d4df1416\") " pod="openshift-image-registry/image-registry-65f68df599-zw6jf"
Jul 16 11:04:05.977095 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.976737    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2ae4a146-99bf-422f-982f-0dab385b060b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-9ztqs\" (UID: \"2ae4a146-99bf-422f-982f-0dab385b060b\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-9ztqs"
Jul 16 11:04:05.977095 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.976764    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f64f15a3-5a9f-4737-bc4f-73d5ec311e41-trusted-ca\") pod \"console-operator-7866497494-9lm5s\" (UID: \"f64f15a3-5a9f-4737-bc4f-73d5ec311e41\") " pod="openshift-console-operator/console-operator-7866497494-9lm5s"
Jul 16 11:04:05.977095 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.976793    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c839c027-e922-48f5-9940-d293b38aef05-serving-cert\") pod \"insights-operator-68d9dd94cf-lx49b\" (UID: \"c839c027-e922-48f5-9940-d293b38aef05\") " pod="openshift-insights/insights-operator-68d9dd94cf-lx49b"
Jul 16 11:04:05.977095 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.976816    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8n8zc\" (UniqueName: \"kubernetes.io/projected/921e629b-d5a3-4bf6-8bad-0131d0a812ea-kube-api-access-8n8zc\") pod \"dns-default-5649t\" (UID: \"921e629b-d5a3-4bf6-8bad-0131d0a812ea\") " pod="openshift-dns/dns-default-5649t"
Jul 16 11:04:05.977095 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.976840    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/543513a9-883b-4f90-8478-eb51b3186af1-config\") pod \"service-ca-operator-7695fc6c8c-7gl88\" (UID: \"543513a9-883b-4f90-8478-eb51b3186af1\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-7gl88"
Jul 16 11:04:05.977095 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.976868    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n9f56\" (UniqueName: \"kubernetes.io/projected/e1f38c54-19be-4f6f-872b-39093678a7ad-kube-api-access-n9f56\") pod \"kube-storage-version-migrator-operator-799cb7f958-85d88\" (UID: \"e1f38c54-19be-4f6f-872b-39093678a7ad\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-85d88"
Jul 16 11:04:05.977095 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.976902    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/6a4b5dde-f664-4d97-889e-bfe4d4df1416-image-registry-private-configuration\") pod \"image-registry-65f68df599-zw6jf\" (UID: \"6a4b5dde-f664-4d97-889e-bfe4d4df1416\") " pod="openshift-image-registry/image-registry-65f68df599-zw6jf"
Jul 16 11:04:05.977095 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.976930    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f64f15a3-5a9f-4737-bc4f-73d5ec311e41-config\") pod \"console-operator-7866497494-9lm5s\" (UID: \"f64f15a3-5a9f-4737-bc4f-73d5ec311e41\") " pod="openshift-console-operator/console-operator-7866497494-9lm5s"
Jul 16 11:04:05.977095 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.976960    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/c839c027-e922-48f5-9940-d293b38aef05-snapshots\") pod \"insights-operator-68d9dd94cf-lx49b\" (UID: \"c839c027-e922-48f5-9940-d293b38aef05\") " pod="openshift-insights/insights-operator-68d9dd94cf-lx49b"
Jul 16 11:04:05.977874 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.976983    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e1f38c54-19be-4f6f-872b-39093678a7ad-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-85d88\" (UID: \"e1f38c54-19be-4f6f-872b-39093678a7ad\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-85d88"
Jul 16 11:04:05.977874 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.977008    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sb2sl\" (UniqueName: \"kubernetes.io/projected/543513a9-883b-4f90-8478-eb51b3186af1-kube-api-access-sb2sl\") pod \"service-ca-operator-7695fc6c8c-7gl88\" (UID: \"543513a9-883b-4f90-8478-eb51b3186af1\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-7gl88"
Jul 16 11:04:05.977874 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.977036    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/921e629b-d5a3-4bf6-8bad-0131d0a812ea-config-volume\") pod \"dns-default-5649t\" (UID: \"921e629b-d5a3-4bf6-8bad-0131d0a812ea\") " pod="openshift-dns/dns-default-5649t"
Jul 16 11:04:05.977874 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.977045    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f64f15a3-5a9f-4737-bc4f-73d5ec311e41-serving-cert\") pod \"console-operator-7866497494-9lm5s\" (UID: \"f64f15a3-5a9f-4737-bc4f-73d5ec311e41\") " pod="openshift-console-operator/console-operator-7866497494-9lm5s"
Jul 16 11:04:05.977874 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.977071    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/921e629b-d5a3-4bf6-8bad-0131d0a812ea-metrics-tls\") pod \"dns-default-5649t\" (UID: \"921e629b-d5a3-4bf6-8bad-0131d0a812ea\") " pod="openshift-dns/dns-default-5649t"
Jul 16 11:04:05.977874 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.977091    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/6a4b5dde-f664-4d97-889e-bfe4d4df1416-registry-certificates\") pod \"image-registry-65f68df599-zw6jf\" (UID: \"6a4b5dde-f664-4d97-889e-bfe4d4df1416\") " pod="openshift-image-registry/image-registry-65f68df599-zw6jf"
Jul 16 11:04:05.977874 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.977153    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rkn9z\" (UniqueName: \"kubernetes.io/projected/2ae4a146-99bf-422f-982f-0dab385b060b-kube-api-access-rkn9z\") pod \"cluster-monitoring-operator-5f7f494dd4-9ztqs\" (UID: \"2ae4a146-99bf-422f-982f-0dab385b060b\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-9ztqs"
Jul 16 11:04:05.977874 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.977180    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6a4b5dde-f664-4d97-889e-bfe4d4df1416-trusted-ca\") pod \"image-registry-65f68df599-zw6jf\" (UID: \"6a4b5dde-f664-4d97-889e-bfe4d4df1416\") " pod="openshift-image-registry/image-registry-65f68df599-zw6jf"
Jul 16 11:04:05.977874 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.977201    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/6a4b5dde-f664-4d97-889e-bfe4d4df1416-installation-pull-secrets\") pod \"image-registry-65f68df599-zw6jf\" (UID: \"6a4b5dde-f664-4d97-889e-bfe4d4df1416\") " pod="openshift-image-registry/image-registry-65f68df599-zw6jf"
Jul 16 11:04:05.977874 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.977226    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e1f38c54-19be-4f6f-872b-39093678a7ad-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-85d88\" (UID: \"e1f38c54-19be-4f6f-872b-39093678a7ad\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-85d88"
Jul 16 11:04:05.977874 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.977236    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/2ae4a146-99bf-422f-982f-0dab385b060b-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-9ztqs\" (UID: \"2ae4a146-99bf-422f-982f-0dab385b060b\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-9ztqs"
Jul 16 11:04:05.977874 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.977257    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c839c027-e922-48f5-9940-d293b38aef05-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-lx49b\" (UID: \"c839c027-e922-48f5-9940-d293b38aef05\") " pod="openshift-insights/insights-operator-68d9dd94cf-lx49b"
Jul 16 11:04:05.977874 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.977283    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6n4lk\" (UniqueName: \"kubernetes.io/projected/ec3eb24d-8f3c-40b7-bfe9-a34fccdc20cd-kube-api-access-6n4lk\") pod \"router-default-76cb9bc7db-x2cnj\" (UID: \"ec3eb24d-8f3c-40b7-bfe9-a34fccdc20cd\") " pod="openshift-ingress/router-default-76cb9bc7db-x2cnj"
Jul 16 11:04:05.977874 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.977292    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/6a4b5dde-f664-4d97-889e-bfe4d4df1416-ca-trust-extracted\") pod \"image-registry-65f68df599-zw6jf\" (UID: \"6a4b5dde-f664-4d97-889e-bfe4d4df1416\") " pod="openshift-image-registry/image-registry-65f68df599-zw6jf"
Jul 16 11:04:05.977874 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.977324    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/ec3eb24d-8f3c-40b7-bfe9-a34fccdc20cd-stats-auth\") pod \"router-default-76cb9bc7db-x2cnj\" (UID: \"ec3eb24d-8f3c-40b7-bfe9-a34fccdc20cd\") " pod="openshift-ingress/router-default-76cb9bc7db-x2cnj"
Jul 16 11:04:05.977874 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.977362    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ec3eb24d-8f3c-40b7-bfe9-a34fccdc20cd-service-ca-bundle\") pod \"router-default-76cb9bc7db-x2cnj\" (UID: \"ec3eb24d-8f3c-40b7-bfe9-a34fccdc20cd\") " pod="openshift-ingress/router-default-76cb9bc7db-x2cnj"
Jul 16 11:04:05.978524 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.977384    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/543513a9-883b-4f90-8478-eb51b3186af1-serving-cert\") pod \"service-ca-operator-7695fc6c8c-7gl88\" (UID: \"543513a9-883b-4f90-8478-eb51b3186af1\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-7gl88"
Jul 16 11:04:05.978524 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.977406    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6a4b5dde-f664-4d97-889e-bfe4d4df1416-registry-tls\") pod \"image-registry-65f68df599-zw6jf\" (UID: \"6a4b5dde-f664-4d97-889e-bfe4d4df1416\") " pod="openshift-image-registry/image-registry-65f68df599-zw6jf"
Jul 16 11:04:05.978524 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:04:05.977520    2583 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 16 11:04:05.978524 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.977519    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c839c027-e922-48f5-9940-d293b38aef05-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-lx49b\" (UID: \"c839c027-e922-48f5-9940-d293b38aef05\") " pod="openshift-insights/insights-operator-68d9dd94cf-lx49b"
Jul 16 11:04:05.978524 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:04:05.977535    2583 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-65f68df599-zw6jf: secret "image-registry-tls" not found
Jul 16 11:04:05.978524 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.977558    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/921e629b-d5a3-4bf6-8bad-0131d0a812ea-tmp-dir\") pod \"dns-default-5649t\" (UID: \"921e629b-d5a3-4bf6-8bad-0131d0a812ea\") " pod="openshift-dns/dns-default-5649t"
Jul 16 11:04:05.978524 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:04:05.977624    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6a4b5dde-f664-4d97-889e-bfe4d4df1416-registry-tls podName:6a4b5dde-f664-4d97-889e-bfe4d4df1416 nodeName:}" failed. No retries permitted until 2026-07-16 11:04:06.477588662 +0000 UTC m=+39.111810134 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/6a4b5dde-f664-4d97-889e-bfe4d4df1416-registry-tls") pod "image-registry-65f68df599-zw6jf" (UID: "6a4b5dde-f664-4d97-889e-bfe4d4df1416") : secret "image-registry-tls" not found
Jul 16 11:04:05.978524 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:04:05.977700    2583 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 16 11:04:05.978524 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:04:05.977738    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ec3eb24d-8f3c-40b7-bfe9-a34fccdc20cd-metrics-certs podName:ec3eb24d-8f3c-40b7-bfe9-a34fccdc20cd nodeName:}" failed. No retries permitted until 2026-07-16 11:04:06.477726661 +0000 UTC m=+39.111948124 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ec3eb24d-8f3c-40b7-bfe9-a34fccdc20cd-metrics-certs") pod "router-default-76cb9bc7db-x2cnj" (UID: "ec3eb24d-8f3c-40b7-bfe9-a34fccdc20cd") : secret "router-metrics-certs-default" not found
Jul 16 11:04:05.978524 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.978016    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c839c027-e922-48f5-9940-d293b38aef05-tmp\") pod \"insights-operator-68d9dd94cf-lx49b\" (UID: \"c839c027-e922-48f5-9940-d293b38aef05\") " pod="openshift-insights/insights-operator-68d9dd94cf-lx49b"
Jul 16 11:04:05.978524 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:04:05.978049    2583 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 16 11:04:05.978524 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:04:05.978142    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ae4a146-99bf-422f-982f-0dab385b060b-cluster-monitoring-operator-tls podName:2ae4a146-99bf-422f-982f-0dab385b060b nodeName:}" failed. No retries permitted until 2026-07-16 11:04:06.47812662 +0000 UTC m=+39.112348071 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/2ae4a146-99bf-422f-982f-0dab385b060b-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-9ztqs" (UID: "2ae4a146-99bf-422f-982f-0dab385b060b") : secret "cluster-monitoring-operator-tls" not found
Jul 16 11:04:05.978524 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.978193    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f64f15a3-5a9f-4737-bc4f-73d5ec311e41-config\") pod \"console-operator-7866497494-9lm5s\" (UID: \"f64f15a3-5a9f-4737-bc4f-73d5ec311e41\") " pod="openshift-console-operator/console-operator-7866497494-9lm5s"
Jul 16 11:04:05.978524 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.978349    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f64f15a3-5a9f-4737-bc4f-73d5ec311e41-trusted-ca\") pod \"console-operator-7866497494-9lm5s\" (UID: \"f64f15a3-5a9f-4737-bc4f-73d5ec311e41\") " pod="openshift-console-operator/console-operator-7866497494-9lm5s"
Jul 16 11:04:05.978524 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.978520    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/543513a9-883b-4f90-8478-eb51b3186af1-config\") pod \"service-ca-operator-7695fc6c8c-7gl88\" (UID: \"543513a9-883b-4f90-8478-eb51b3186af1\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-7gl88"
Jul 16 11:04:05.979181 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:04:05.978644    2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 16 11:04:05.979181 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.978647    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/c839c027-e922-48f5-9940-d293b38aef05-snapshots\") pod \"insights-operator-68d9dd94cf-lx49b\" (UID: \"c839c027-e922-48f5-9940-d293b38aef05\") " pod="openshift-insights/insights-operator-68d9dd94cf-lx49b"
Jul 16 11:04:05.979181 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:04:05.978687    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/921e629b-d5a3-4bf6-8bad-0131d0a812ea-metrics-tls podName:921e629b-d5a3-4bf6-8bad-0131d0a812ea nodeName:}" failed. No retries permitted until 2026-07-16 11:04:06.478673171 +0000 UTC m=+39.112894620 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/921e629b-d5a3-4bf6-8bad-0131d0a812ea-metrics-tls") pod "dns-default-5649t" (UID: "921e629b-d5a3-4bf6-8bad-0131d0a812ea") : secret "dns-default-metrics-tls" not found
Jul 16 11:04:05.979306 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.979189    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e1f38c54-19be-4f6f-872b-39093678a7ad-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-85d88\" (UID: \"e1f38c54-19be-4f6f-872b-39093678a7ad\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-85d88"
Jul 16 11:04:05.979582 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.979559    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/6a4b5dde-f664-4d97-889e-bfe4d4df1416-registry-certificates\") pod \"image-registry-65f68df599-zw6jf\" (UID: \"6a4b5dde-f664-4d97-889e-bfe4d4df1416\") " pod="openshift-image-registry/image-registry-65f68df599-zw6jf"
Jul 16 11:04:05.980438 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.980405    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c839c027-e922-48f5-9940-d293b38aef05-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-lx49b\" (UID: \"c839c027-e922-48f5-9940-d293b38aef05\") " pod="openshift-insights/insights-operator-68d9dd94cf-lx49b"
Jul 16 11:04:05.980663 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:04:05.980645    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ec3eb24d-8f3c-40b7-bfe9-a34fccdc20cd-service-ca-bundle podName:ec3eb24d-8f3c-40b7-bfe9-a34fccdc20cd nodeName:}" failed. No retries permitted until 2026-07-16 11:04:06.480037167 +0000 UTC m=+39.114258632 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/ec3eb24d-8f3c-40b7-bfe9-a34fccdc20cd-service-ca-bundle") pod "router-default-76cb9bc7db-x2cnj" (UID: "ec3eb24d-8f3c-40b7-bfe9-a34fccdc20cd") : configmap references non-existent config key: service-ca.crt
Jul 16 11:04:05.981091 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.981064    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6a4b5dde-f664-4d97-889e-bfe4d4df1416-trusted-ca\") pod \"image-registry-65f68df599-zw6jf\" (UID: \"6a4b5dde-f664-4d97-889e-bfe4d4df1416\") " pod="openshift-image-registry/image-registry-65f68df599-zw6jf"
Jul 16 11:04:05.982885 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.982856    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/ec3eb24d-8f3c-40b7-bfe9-a34fccdc20cd-default-certificate\") pod \"router-default-76cb9bc7db-x2cnj\" (UID: \"ec3eb24d-8f3c-40b7-bfe9-a34fccdc20cd\") " pod="openshift-ingress/router-default-76cb9bc7db-x2cnj"
Jul 16 11:04:05.983999 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.983244    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/6a4b5dde-f664-4d97-889e-bfe4d4df1416-installation-pull-secrets\") pod \"image-registry-65f68df599-zw6jf\" (UID: \"6a4b5dde-f664-4d97-889e-bfe4d4df1416\") " pod="openshift-image-registry/image-registry-65f68df599-zw6jf"
Jul 16 11:04:05.983999 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.983425    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e1f38c54-19be-4f6f-872b-39093678a7ad-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-85d88\" (UID: \"e1f38c54-19be-4f6f-872b-39093678a7ad\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-85d88"
Jul 16 11:04:05.983999 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.983516    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c839c027-e922-48f5-9940-d293b38aef05-serving-cert\") pod \"insights-operator-68d9dd94cf-lx49b\" (UID: \"c839c027-e922-48f5-9940-d293b38aef05\") " pod="openshift-insights/insights-operator-68d9dd94cf-lx49b"
Jul 16 11:04:05.983999 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.983935    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8dcffe9f-afd4-4e68-9c2a-a6d824df311e-cert\") pod \"ingress-canary-4mhsn\" (UID: \"8dcffe9f-afd4-4e68-9c2a-a6d824df311e\") " pod="openshift-ingress-canary/ingress-canary-4mhsn"
Jul 16 11:04:05.983999 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.983956    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/ec3eb24d-8f3c-40b7-bfe9-a34fccdc20cd-stats-auth\") pod \"router-default-76cb9bc7db-x2cnj\" (UID: \"ec3eb24d-8f3c-40b7-bfe9-a34fccdc20cd\") " pod="openshift-ingress/router-default-76cb9bc7db-x2cnj"
Jul 16 11:04:05.984312 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.984280    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/543513a9-883b-4f90-8478-eb51b3186af1-serving-cert\") pod \"service-ca-operator-7695fc6c8c-7gl88\" (UID: \"543513a9-883b-4f90-8478-eb51b3186af1\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-7gl88"
Jul 16 11:04:05.984373 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.984354    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f64f15a3-5a9f-4737-bc4f-73d5ec311e41-serving-cert\") pod \"console-operator-7866497494-9lm5s\" (UID: \"f64f15a3-5a9f-4737-bc4f-73d5ec311e41\") " pod="openshift-console-operator/console-operator-7866497494-9lm5s"
Jul 16 11:04:05.984922 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.984903    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/6a4b5dde-f664-4d97-889e-bfe4d4df1416-image-registry-private-configuration\") pod \"image-registry-65f68df599-zw6jf\" (UID: \"6a4b5dde-f664-4d97-889e-bfe4d4df1416\") " pod="openshift-image-registry/image-registry-65f68df599-zw6jf"
Jul 16 11:04:05.988441 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.988416    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vgps7\" (UniqueName: \"kubernetes.io/projected/ca7a1d19-213d-4ce9-8a34-4a93ddcab230-kube-api-access-vgps7\") pod \"volume-data-source-validator-78495597f6-d496q\" (UID: \"ca7a1d19-213d-4ce9-8a34-4a93ddcab230\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-d496q"
Jul 16 11:04:05.988545 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.988448    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sc6xs\" (UniqueName: \"kubernetes.io/projected/8dcffe9f-afd4-4e68-9c2a-a6d824df311e-kube-api-access-sc6xs\") pod \"ingress-canary-4mhsn\" (UID: \"8dcffe9f-afd4-4e68-9c2a-a6d824df311e\") " pod="openshift-ingress-canary/ingress-canary-4mhsn"
Jul 16 11:04:05.988623 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.988558    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f7hst\" (UniqueName: \"kubernetes.io/projected/369fdbca-23e1-4389-9b69-4f09463fc7bc-kube-api-access-f7hst\") pod \"network-check-source-869dcfcdcd-7d8g4\" (UID: \"369fdbca-23e1-4389-9b69-4f09463fc7bc\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-7d8g4"
Jul 16 11:04:05.990410 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.989814    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n9f56\" (UniqueName: \"kubernetes.io/projected/e1f38c54-19be-4f6f-872b-39093678a7ad-kube-api-access-n9f56\") pod \"kube-storage-version-migrator-operator-799cb7f958-85d88\" (UID: \"e1f38c54-19be-4f6f-872b-39093678a7ad\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-85d88"
Jul 16 11:04:05.990410 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.990362    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f5zqf\" (UniqueName: \"kubernetes.io/projected/f64f15a3-5a9f-4737-bc4f-73d5ec311e41-kube-api-access-f5zqf\") pod \"console-operator-7866497494-9lm5s\" (UID: \"f64f15a3-5a9f-4737-bc4f-73d5ec311e41\") " pod="openshift-console-operator/console-operator-7866497494-9lm5s"
Jul 16 11:04:05.990410 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.990381    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2mxl4\" (UniqueName: \"kubernetes.io/projected/6a4b5dde-f664-4d97-889e-bfe4d4df1416-kube-api-access-2mxl4\") pod \"image-registry-65f68df599-zw6jf\" (UID: \"6a4b5dde-f664-4d97-889e-bfe4d4df1416\") " pod="openshift-image-registry/image-registry-65f68df599-zw6jf"
Jul 16 11:04:05.990702 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.990378    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rkn9z\" (UniqueName: \"kubernetes.io/projected/2ae4a146-99bf-422f-982f-0dab385b060b-kube-api-access-rkn9z\") pod \"cluster-monitoring-operator-5f7f494dd4-9ztqs\" (UID: \"2ae4a146-99bf-422f-982f-0dab385b060b\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-9ztqs"
Jul 16 11:04:05.991855 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.991831    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8n8zc\" (UniqueName: \"kubernetes.io/projected/921e629b-d5a3-4bf6-8bad-0131d0a812ea-kube-api-access-8n8zc\") pod \"dns-default-5649t\" (UID: \"921e629b-d5a3-4bf6-8bad-0131d0a812ea\") " pod="openshift-dns/dns-default-5649t"
Jul 16 11:04:05.991949 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.991850    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6a4b5dde-f664-4d97-889e-bfe4d4df1416-bound-sa-token\") pod \"image-registry-65f68df599-zw6jf\" (UID: \"6a4b5dde-f664-4d97-889e-bfe4d4df1416\") " pod="openshift-image-registry/image-registry-65f68df599-zw6jf"
Jul 16 11:04:05.991949 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.991933    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8tgg7\" (UniqueName: \"kubernetes.io/projected/628849b1-f592-48a1-af55-d6f78351be63-kube-api-access-8tgg7\") pod \"cluster-samples-operator-65db858894-982zd\" (UID: \"628849b1-f592-48a1-af55-d6f78351be63\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-982zd"
Jul 16 11:04:05.992775 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.992660    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sb2sl\" (UniqueName: \"kubernetes.io/projected/543513a9-883b-4f90-8478-eb51b3186af1-kube-api-access-sb2sl\") pod \"service-ca-operator-7695fc6c8c-7gl88\" (UID: \"543513a9-883b-4f90-8478-eb51b3186af1\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-7gl88"
Jul 16 11:04:05.993636 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.993517    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6n4lk\" (UniqueName: \"kubernetes.io/projected/ec3eb24d-8f3c-40b7-bfe9-a34fccdc20cd-kube-api-access-6n4lk\") pod \"router-default-76cb9bc7db-x2cnj\" (UID: \"ec3eb24d-8f3c-40b7-bfe9-a34fccdc20cd\") " pod="openshift-ingress/router-default-76cb9bc7db-x2cnj"
Jul 16 11:04:05.993760 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:05.993746    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-44gnb\" (UniqueName: \"kubernetes.io/projected/c839c027-e922-48f5-9940-d293b38aef05-kube-api-access-44gnb\") pod \"insights-operator-68d9dd94cf-lx49b\" (UID: \"c839c027-e922-48f5-9940-d293b38aef05\") " pod="openshift-insights/insights-operator-68d9dd94cf-lx49b"
Jul 16 11:04:06.027758 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:06.027716    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-d496q"
Jul 16 11:04:06.077579 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:06.077542    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-68d9dd94cf-lx49b"
Jul 16 11:04:06.085519 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:06.085490    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-85d88"
Jul 16 11:04:06.107023 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:06.106985    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7866497494-9lm5s"
Jul 16 11:04:06.114244 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:06.114209    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-7d8g4"
Jul 16 11:04:06.121408 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:06.121241    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-7gl88"
Jul 16 11:04:06.126236 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:06.126204    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-4mhsn"
Jul 16 11:04:06.325817 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:06.325739    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qxpqt_c24aaddd-b44c-4155-a7c5-fadb281f9b39/konnectivity-agent/0.log" limit=1024
Jul 16 11:04:06.480865 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:06.480829    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ec3eb24d-8f3c-40b7-bfe9-a34fccdc20cd-service-ca-bundle\") pod \"router-default-76cb9bc7db-x2cnj\" (UID: \"ec3eb24d-8f3c-40b7-bfe9-a34fccdc20cd\") " pod="openshift-ingress/router-default-76cb9bc7db-x2cnj"
Jul 16 11:04:06.481042 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:06.480878    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6a4b5dde-f664-4d97-889e-bfe4d4df1416-registry-tls\") pod \"image-registry-65f68df599-zw6jf\" (UID: \"6a4b5dde-f664-4d97-889e-bfe4d4df1416\") " pod="openshift-image-registry/image-registry-65f68df599-zw6jf"
Jul 16 11:04:06.481042 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:06.480908    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ec3eb24d-8f3c-40b7-bfe9-a34fccdc20cd-metrics-certs\") pod \"router-default-76cb9bc7db-x2cnj\" (UID: \"ec3eb24d-8f3c-40b7-bfe9-a34fccdc20cd\") " pod="openshift-ingress/router-default-76cb9bc7db-x2cnj"
Jul 16 11:04:06.481042 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:06.480935    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/628849b1-f592-48a1-af55-d6f78351be63-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-982zd\" (UID: \"628849b1-f592-48a1-af55-d6f78351be63\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-982zd"
Jul 16 11:04:06.481183 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:04:06.481059    2583 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 16 11:04:06.481183 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:04:06.481080    2583 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-65f68df599-zw6jf: secret "image-registry-tls" not found
Jul 16 11:04:06.481183 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:04:06.481064    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ec3eb24d-8f3c-40b7-bfe9-a34fccdc20cd-service-ca-bundle podName:ec3eb24d-8f3c-40b7-bfe9-a34fccdc20cd nodeName:}" failed. No retries permitted until 2026-07-16 11:04:07.481040012 +0000 UTC m=+40.115261444 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/ec3eb24d-8f3c-40b7-bfe9-a34fccdc20cd-service-ca-bundle") pod "router-default-76cb9bc7db-x2cnj" (UID: "ec3eb24d-8f3c-40b7-bfe9-a34fccdc20cd") : configmap references non-existent config key: service-ca.crt
Jul 16 11:04:06.481183 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:04:06.481064    2583 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 16 11:04:06.481183 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:06.481153    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2ae4a146-99bf-422f-982f-0dab385b060b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-9ztqs\" (UID: \"2ae4a146-99bf-422f-982f-0dab385b060b\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-9ztqs"
Jul 16 11:04:06.481183 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:04:06.481064    2583 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 16 11:04:06.481441 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:04:06.481195    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/628849b1-f592-48a1-af55-d6f78351be63-samples-operator-tls podName:628849b1-f592-48a1-af55-d6f78351be63 nodeName:}" failed. No retries permitted until 2026-07-16 11:04:07.481179485 +0000 UTC m=+40.115400934 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/628849b1-f592-48a1-af55-d6f78351be63-samples-operator-tls") pod "cluster-samples-operator-65db858894-982zd" (UID: "628849b1-f592-48a1-af55-d6f78351be63") : secret "samples-operator-tls" not found
Jul 16 11:04:06.481441 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:04:06.481206    2583 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 16 11:04:06.481441 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:04:06.481218    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ec3eb24d-8f3c-40b7-bfe9-a34fccdc20cd-metrics-certs podName:ec3eb24d-8f3c-40b7-bfe9-a34fccdc20cd nodeName:}" failed. No retries permitted until 2026-07-16 11:04:07.481208315 +0000 UTC m=+40.115429763 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ec3eb24d-8f3c-40b7-bfe9-a34fccdc20cd-metrics-certs") pod "router-default-76cb9bc7db-x2cnj" (UID: "ec3eb24d-8f3c-40b7-bfe9-a34fccdc20cd") : secret "router-metrics-certs-default" not found
Jul 16 11:04:06.481441 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:04:06.481235    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6a4b5dde-f664-4d97-889e-bfe4d4df1416-registry-tls podName:6a4b5dde-f664-4d97-889e-bfe4d4df1416 nodeName:}" failed. No retries permitted until 2026-07-16 11:04:07.481227013 +0000 UTC m=+40.115448458 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/6a4b5dde-f664-4d97-889e-bfe4d4df1416-registry-tls") pod "image-registry-65f68df599-zw6jf" (UID: "6a4b5dde-f664-4d97-889e-bfe4d4df1416") : secret "image-registry-tls" not found
Jul 16 11:04:06.481441 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:04:06.481254    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ae4a146-99bf-422f-982f-0dab385b060b-cluster-monitoring-operator-tls podName:2ae4a146-99bf-422f-982f-0dab385b060b nodeName:}" failed. No retries permitted until 2026-07-16 11:04:07.481241076 +0000 UTC m=+40.115462511 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/2ae4a146-99bf-422f-982f-0dab385b060b-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-9ztqs" (UID: "2ae4a146-99bf-422f-982f-0dab385b060b") : secret "cluster-monitoring-operator-tls" not found
Jul 16 11:04:06.481441 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:06.481305    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/921e629b-d5a3-4bf6-8bad-0131d0a812ea-metrics-tls\") pod \"dns-default-5649t\" (UID: \"921e629b-d5a3-4bf6-8bad-0131d0a812ea\") " pod="openshift-dns/dns-default-5649t"
Jul 16 11:04:06.481441 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:04:06.481426    2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 16 11:04:06.481695 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:04:06.481470    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/921e629b-d5a3-4bf6-8bad-0131d0a812ea-metrics-tls podName:921e629b-d5a3-4bf6-8bad-0131d0a812ea nodeName:}" failed. No retries permitted until 2026-07-16 11:04:07.481457935 +0000 UTC m=+40.115679378 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/921e629b-d5a3-4bf6-8bad-0131d0a812ea-metrics-tls") pod "dns-default-5649t" (UID: "921e629b-d5a3-4bf6-8bad-0131d0a812ea") : secret "dns-default-metrics-tls" not found
Jul 16 11:04:06.763778 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:06.763499    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-7gl88"]
Jul 16 11:04:06.766737 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:06.766585    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-85d88"]
Jul 16 11:04:06.769289 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:06.769259    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-4mhsn"]
Jul 16 11:04:06.783202 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:06.783170    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-lx49b"]
Jul 16 11:04:06.794657 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:06.794627    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7866497494-9lm5s"]
Jul 16 11:04:06.797161 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:06.797136    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-d496q"]
Jul 16 11:04:06.806560 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:06.806527    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-7d8g4"]
Jul 16 11:04:06.810090 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:04:06.810056    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8dcffe9f_afd4_4e68_9c2a_a6d824df311e.slice/crio-7f02c05d56b0c2223202b4ed7b14f140f488e896edf8585fcfcc5a87a3138cca WatchSource:0}: Error finding container 7f02c05d56b0c2223202b4ed7b14f140f488e896edf8585fcfcc5a87a3138cca: Status 404 returned error can't find the container with id 7f02c05d56b0c2223202b4ed7b14f140f488e896edf8585fcfcc5a87a3138cca
Jul 16 11:04:06.814524 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:04:06.814367    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf64f15a3_5a9f_4737_bc4f_73d5ec311e41.slice/crio-c34d7ef88cdfe997144057b920ae9168c167b9e1b20eaaa8ac6c73df96b9f734 WatchSource:0}: Error finding container c34d7ef88cdfe997144057b920ae9168c167b9e1b20eaaa8ac6c73df96b9f734: Status 404 returned error can't find the container with id c34d7ef88cdfe997144057b920ae9168c167b9e1b20eaaa8ac6c73df96b9f734
Jul 16 11:04:07.223062 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:07.222994    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hlm2s"
Jul 16 11:04:07.223468 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:07.223008    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qcl5d"
Jul 16 11:04:07.223468 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:07.223129    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rc7w4"
Jul 16 11:04:07.227109 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:07.226783    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 16 11:04:07.227109 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:07.226805    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 16 11:04:07.227109 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:07.226805    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-rdfxb\""
Jul 16 11:04:07.227109 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:07.226856    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-49sx5\""
Jul 16 11:04:07.491531 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:07.491455    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/921e629b-d5a3-4bf6-8bad-0131d0a812ea-metrics-tls\") pod \"dns-default-5649t\" (UID: \"921e629b-d5a3-4bf6-8bad-0131d0a812ea\") " pod="openshift-dns/dns-default-5649t"
Jul 16 11:04:07.491531 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:07.491517    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ec3eb24d-8f3c-40b7-bfe9-a34fccdc20cd-service-ca-bundle\") pod \"router-default-76cb9bc7db-x2cnj\" (UID: \"ec3eb24d-8f3c-40b7-bfe9-a34fccdc20cd\") " pod="openshift-ingress/router-default-76cb9bc7db-x2cnj"
Jul 16 11:04:07.492320 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:07.491537    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6a4b5dde-f664-4d97-889e-bfe4d4df1416-registry-tls\") pod \"image-registry-65f68df599-zw6jf\" (UID: \"6a4b5dde-f664-4d97-889e-bfe4d4df1416\") " pod="openshift-image-registry/image-registry-65f68df599-zw6jf"
Jul 16 11:04:07.492320 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:07.491557    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ec3eb24d-8f3c-40b7-bfe9-a34fccdc20cd-metrics-certs\") pod \"router-default-76cb9bc7db-x2cnj\" (UID: \"ec3eb24d-8f3c-40b7-bfe9-a34fccdc20cd\") " pod="openshift-ingress/router-default-76cb9bc7db-x2cnj"
Jul 16 11:04:07.492320 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:07.491581    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/628849b1-f592-48a1-af55-d6f78351be63-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-982zd\" (UID: \"628849b1-f592-48a1-af55-d6f78351be63\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-982zd"
Jul 16 11:04:07.492320 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:04:07.491687    2583 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 16 11:04:07.492320 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:07.491710    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2ae4a146-99bf-422f-982f-0dab385b060b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-9ztqs\" (UID: \"2ae4a146-99bf-422f-982f-0dab385b060b\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-9ztqs"
Jul 16 11:04:07.492320 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:04:07.491734    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/628849b1-f592-48a1-af55-d6f78351be63-samples-operator-tls podName:628849b1-f592-48a1-af55-d6f78351be63 nodeName:}" failed. No retries permitted until 2026-07-16 11:04:09.491720002 +0000 UTC m=+42.125941438 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/628849b1-f592-48a1-af55-d6f78351be63-samples-operator-tls") pod "cluster-samples-operator-65db858894-982zd" (UID: "628849b1-f592-48a1-af55-d6f78351be63") : secret "samples-operator-tls" not found
Jul 16 11:04:07.492320 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:04:07.491744    2583 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 16 11:04:07.492320 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:04:07.491763    2583 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-65f68df599-zw6jf: secret "image-registry-tls" not found
Jul 16 11:04:07.492320 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:04:07.491796    2583 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 16 11:04:07.492320 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:04:07.491815    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6a4b5dde-f664-4d97-889e-bfe4d4df1416-registry-tls podName:6a4b5dde-f664-4d97-889e-bfe4d4df1416 nodeName:}" failed. No retries permitted until 2026-07-16 11:04:09.491798909 +0000 UTC m=+42.126020349 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/6a4b5dde-f664-4d97-889e-bfe4d4df1416-registry-tls") pod "image-registry-65f68df599-zw6jf" (UID: "6a4b5dde-f664-4d97-889e-bfe4d4df1416") : secret "image-registry-tls" not found
Jul 16 11:04:07.492320 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:04:07.491844    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ae4a146-99bf-422f-982f-0dab385b060b-cluster-monitoring-operator-tls podName:2ae4a146-99bf-422f-982f-0dab385b060b nodeName:}" failed. No retries permitted until 2026-07-16 11:04:09.491829999 +0000 UTC m=+42.126051434 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/2ae4a146-99bf-422f-982f-0dab385b060b-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-9ztqs" (UID: "2ae4a146-99bf-422f-982f-0dab385b060b") : secret "cluster-monitoring-operator-tls" not found
Jul 16 11:04:07.492320 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:04:07.491800    2583 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 16 11:04:07.492320 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:04:07.491886    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ec3eb24d-8f3c-40b7-bfe9-a34fccdc20cd-metrics-certs podName:ec3eb24d-8f3c-40b7-bfe9-a34fccdc20cd nodeName:}" failed. No retries permitted until 2026-07-16 11:04:09.491875054 +0000 UTC m=+42.126096493 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ec3eb24d-8f3c-40b7-bfe9-a34fccdc20cd-metrics-certs") pod "router-default-76cb9bc7db-x2cnj" (UID: "ec3eb24d-8f3c-40b7-bfe9-a34fccdc20cd") : secret "router-metrics-certs-default" not found
Jul 16 11:04:07.492320 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:04:07.491900    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ec3eb24d-8f3c-40b7-bfe9-a34fccdc20cd-service-ca-bundle podName:ec3eb24d-8f3c-40b7-bfe9-a34fccdc20cd nodeName:}" failed. No retries permitted until 2026-07-16 11:04:09.491891575 +0000 UTC m=+42.126113008 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/ec3eb24d-8f3c-40b7-bfe9-a34fccdc20cd-service-ca-bundle") pod "router-default-76cb9bc7db-x2cnj" (UID: "ec3eb24d-8f3c-40b7-bfe9-a34fccdc20cd") : configmap references non-existent config key: service-ca.crt
Jul 16 11:04:07.492320 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:04:07.491925    2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 16 11:04:07.493076 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:04:07.491966    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/921e629b-d5a3-4bf6-8bad-0131d0a812ea-metrics-tls podName:921e629b-d5a3-4bf6-8bad-0131d0a812ea nodeName:}" failed. No retries permitted until 2026-07-16 11:04:09.491955219 +0000 UTC m=+42.126176653 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/921e629b-d5a3-4bf6-8bad-0131d0a812ea-metrics-tls") pod "dns-default-5649t" (UID: "921e629b-d5a3-4bf6-8bad-0131d0a812ea") : secret "dns-default-metrics-tls" not found
Jul 16 11:04:07.581655 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:07.581617    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-4mhsn" event={"ID":"8dcffe9f-afd4-4e68-9c2a-a6d824df311e","Type":"ContainerStarted","Data":"7f02c05d56b0c2223202b4ed7b14f140f488e896edf8585fcfcc5a87a3138cca"}
Jul 16 11:04:07.582971 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:07.582943    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-85d88" event={"ID":"e1f38c54-19be-4f6f-872b-39093678a7ad","Type":"ContainerStarted","Data":"338356ad7661e933bdd17c630ec83ef92a059abafeb72e9baae8a76c377bdc72"}
Jul 16 11:04:07.588211 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:07.587698    2583 generic.go:356] "Generic (PLEG): container finished" podID="6fd1e802-78a8-4991-99ee-47cfea84ab62" containerID="3dc3797780cd6e986f093755c9db38cce5ca761b3b5e348b711b7bee7192eb1f" exitCode=0
Jul 16 11:04:07.588211 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:07.587773    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8r89z" event={"ID":"6fd1e802-78a8-4991-99ee-47cfea84ab62","Type":"ContainerDied","Data":"3dc3797780cd6e986f093755c9db38cce5ca761b3b5e348b711b7bee7192eb1f"}
Jul 16 11:04:07.592286 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:07.592223    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-7gl88" event={"ID":"543513a9-883b-4f90-8478-eb51b3186af1","Type":"ContainerStarted","Data":"e7a3075223a7d7ea613571046c4b5a7fbc5ce799fb7bda625080097382ad34fa"}
Jul 16 11:04:07.595104 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:07.595040    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-7d8g4" event={"ID":"369fdbca-23e1-4389-9b69-4f09463fc7bc","Type":"ContainerStarted","Data":"fe4052b1bb6e0efd866ff15be0dd5ea61a49d9dc5b668db5aa13cd254ac48365"}
Jul 16 11:04:07.597550 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:07.597473    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-9lm5s" event={"ID":"f64f15a3-5a9f-4737-bc4f-73d5ec311e41","Type":"ContainerStarted","Data":"c34d7ef88cdfe997144057b920ae9168c167b9e1b20eaaa8ac6c73df96b9f734"}
Jul 16 11:04:07.602241 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:07.602176    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-d496q" event={"ID":"ca7a1d19-213d-4ce9-8a34-4a93ddcab230","Type":"ContainerStarted","Data":"3bc73de104322035aa9a8238ad864c0563d8efdccf69623fa2349977fad2f0c1"}
Jul 16 11:04:07.603713 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:07.603650    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-lx49b" event={"ID":"c839c027-e922-48f5-9940-d293b38aef05","Type":"ContainerStarted","Data":"9284f2c4d4587c6a6b957eec1dd854ab09a5e1ba81d260c8474c86200f906d04"}
Jul 16 11:04:08.615325 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:08.614212    2583 generic.go:356] "Generic (PLEG): container finished" podID="6fd1e802-78a8-4991-99ee-47cfea84ab62" containerID="2e468ad7b48edfb1d9fd74817f5cdb7536064d52b07a08beeb284b2b777944b4" exitCode=0
Jul 16 11:04:08.615325 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:08.614276    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8r89z" event={"ID":"6fd1e802-78a8-4991-99ee-47cfea84ab62","Type":"ContainerDied","Data":"2e468ad7b48edfb1d9fd74817f5cdb7536064d52b07a08beeb284b2b777944b4"}
Jul 16 11:04:08.658900 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:08.658860    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-vgn2h"]
Jul 16 11:04:08.668968 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:08.668937    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-vgn2h"
Jul 16 11:04:08.671949 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:08.671913    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-fn5b5\""
Jul 16 11:04:08.672223 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:08.672204    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Jul 16 11:04:08.672493 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:08.672479    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Jul 16 11:04:08.678885 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:08.678856    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-vgn2h"]
Jul 16 11:04:08.795975 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:08.795938    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qxpqt_c24aaddd-b44c-4155-a7c5-fadb281f9b39/konnectivity-agent/0.log" limit=1024
Jul 16 11:04:08.810424 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:08.810382    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/9fceac27-6fd9-4091-af74-783b21616981-nginx-conf\") pod \"networking-console-plugin-6cf856758c-vgn2h\" (UID: \"9fceac27-6fd9-4091-af74-783b21616981\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-vgn2h"
Jul 16 11:04:08.810584 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:08.810478    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9fceac27-6fd9-4091-af74-783b21616981-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-vgn2h\" (UID: \"9fceac27-6fd9-4091-af74-783b21616981\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-vgn2h"
Jul 16 11:04:08.899516 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:08.899402    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qxpqt_c24aaddd-b44c-4155-a7c5-fadb281f9b39/konnectivity-agent/0.log" limit=1024
Jul 16 11:04:08.911477 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:08.911286    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9fceac27-6fd9-4091-af74-783b21616981-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-vgn2h\" (UID: \"9fceac27-6fd9-4091-af74-783b21616981\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-vgn2h"
Jul 16 11:04:08.911477 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:04:08.911403    2583 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 16 11:04:08.911477 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:04:08.911470    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9fceac27-6fd9-4091-af74-783b21616981-networking-console-plugin-cert podName:9fceac27-6fd9-4091-af74-783b21616981 nodeName:}" failed. No retries permitted until 2026-07-16 11:04:09.411451964 +0000 UTC m=+42.045673399 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/9fceac27-6fd9-4091-af74-783b21616981-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-vgn2h" (UID: "9fceac27-6fd9-4091-af74-783b21616981") : secret "networking-console-plugin-cert" not found
Jul 16 11:04:08.911820 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:08.911501    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/9fceac27-6fd9-4091-af74-783b21616981-nginx-conf\") pod \"networking-console-plugin-6cf856758c-vgn2h\" (UID: \"9fceac27-6fd9-4091-af74-783b21616981\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-vgn2h"
Jul 16 11:04:08.912350 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:08.912309    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/9fceac27-6fd9-4091-af74-783b21616981-nginx-conf\") pod \"networking-console-plugin-6cf856758c-vgn2h\" (UID: \"9fceac27-6fd9-4091-af74-783b21616981\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-vgn2h"
Jul 16 11:04:08.992911 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:08.992645    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qxpqt_c24aaddd-b44c-4155-a7c5-fadb281f9b39/konnectivity-agent/0.log" limit=1024
Jul 16 11:04:09.419074 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:09.419037    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9fceac27-6fd9-4091-af74-783b21616981-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-vgn2h\" (UID: \"9fceac27-6fd9-4091-af74-783b21616981\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-vgn2h"
Jul 16 11:04:09.419263 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:04:09.419222    2583 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 16 11:04:09.419327 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:04:09.419302    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9fceac27-6fd9-4091-af74-783b21616981-networking-console-plugin-cert podName:9fceac27-6fd9-4091-af74-783b21616981 nodeName:}" failed. No retries permitted until 2026-07-16 11:04:10.419280616 +0000 UTC m=+43.053502071 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/9fceac27-6fd9-4091-af74-783b21616981-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-vgn2h" (UID: "9fceac27-6fd9-4091-af74-783b21616981") : secret "networking-console-plugin-cert" not found
Jul 16 11:04:09.519886 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:09.519846    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/921e629b-d5a3-4bf6-8bad-0131d0a812ea-metrics-tls\") pod \"dns-default-5649t\" (UID: \"921e629b-d5a3-4bf6-8bad-0131d0a812ea\") " pod="openshift-dns/dns-default-5649t"
Jul 16 11:04:09.520056 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:09.519947    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ec3eb24d-8f3c-40b7-bfe9-a34fccdc20cd-service-ca-bundle\") pod \"router-default-76cb9bc7db-x2cnj\" (UID: \"ec3eb24d-8f3c-40b7-bfe9-a34fccdc20cd\") " pod="openshift-ingress/router-default-76cb9bc7db-x2cnj"
Jul 16 11:04:09.520056 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:09.519985    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6a4b5dde-f664-4d97-889e-bfe4d4df1416-registry-tls\") pod \"image-registry-65f68df599-zw6jf\" (UID: \"6a4b5dde-f664-4d97-889e-bfe4d4df1416\") " pod="openshift-image-registry/image-registry-65f68df599-zw6jf"
Jul 16 11:04:09.520056 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:09.520016    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ec3eb24d-8f3c-40b7-bfe9-a34fccdc20cd-metrics-certs\") pod \"router-default-76cb9bc7db-x2cnj\" (UID: \"ec3eb24d-8f3c-40b7-bfe9-a34fccdc20cd\") " pod="openshift-ingress/router-default-76cb9bc7db-x2cnj"
Jul 16 11:04:09.520056 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:04:09.520018    2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 16 11:04:09.520056 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:09.520052    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/628849b1-f592-48a1-af55-d6f78351be63-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-982zd\" (UID: \"628849b1-f592-48a1-af55-d6f78351be63\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-982zd"
Jul 16 11:04:09.520312 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:04:09.520092    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/921e629b-d5a3-4bf6-8bad-0131d0a812ea-metrics-tls podName:921e629b-d5a3-4bf6-8bad-0131d0a812ea nodeName:}" failed. No retries permitted until 2026-07-16 11:04:13.520072496 +0000 UTC m=+46.154293933 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/921e629b-d5a3-4bf6-8bad-0131d0a812ea-metrics-tls") pod "dns-default-5649t" (UID: "921e629b-d5a3-4bf6-8bad-0131d0a812ea") : secret "dns-default-metrics-tls" not found
Jul 16 11:04:09.520312 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:04:09.520112    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ec3eb24d-8f3c-40b7-bfe9-a34fccdc20cd-service-ca-bundle podName:ec3eb24d-8f3c-40b7-bfe9-a34fccdc20cd nodeName:}" failed. No retries permitted until 2026-07-16 11:04:13.520102385 +0000 UTC m=+46.154323823 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/ec3eb24d-8f3c-40b7-bfe9-a34fccdc20cd-service-ca-bundle") pod "router-default-76cb9bc7db-x2cnj" (UID: "ec3eb24d-8f3c-40b7-bfe9-a34fccdc20cd") : configmap references non-existent config key: service-ca.crt
Jul 16 11:04:09.520312 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:04:09.520156    2583 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 16 11:04:09.520312 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:04:09.520175    2583 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-65f68df599-zw6jf: secret "image-registry-tls" not found
Jul 16 11:04:09.520312 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:04:09.520209    2583 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 16 11:04:09.520312 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:04:09.520162    2583 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 16 11:04:09.520312 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:09.520170    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2ae4a146-99bf-422f-982f-0dab385b060b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-9ztqs\" (UID: \"2ae4a146-99bf-422f-982f-0dab385b060b\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-9ztqs"
Jul 16 11:04:09.520312 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:04:09.520227    2583 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 16 11:04:09.520312 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:04:09.520219    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6a4b5dde-f664-4d97-889e-bfe4d4df1416-registry-tls podName:6a4b5dde-f664-4d97-889e-bfe4d4df1416 nodeName:}" failed. No retries permitted until 2026-07-16 11:04:13.52020401 +0000 UTC m=+46.154425442 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/6a4b5dde-f664-4d97-889e-bfe4d4df1416-registry-tls") pod "image-registry-65f68df599-zw6jf" (UID: "6a4b5dde-f664-4d97-889e-bfe4d4df1416") : secret "image-registry-tls" not found
Jul 16 11:04:09.520312 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:04:09.520265    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ec3eb24d-8f3c-40b7-bfe9-a34fccdc20cd-metrics-certs podName:ec3eb24d-8f3c-40b7-bfe9-a34fccdc20cd nodeName:}" failed. No retries permitted until 2026-07-16 11:04:13.520251582 +0000 UTC m=+46.154473016 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ec3eb24d-8f3c-40b7-bfe9-a34fccdc20cd-metrics-certs") pod "router-default-76cb9bc7db-x2cnj" (UID: "ec3eb24d-8f3c-40b7-bfe9-a34fccdc20cd") : secret "router-metrics-certs-default" not found
Jul 16 11:04:09.520312 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:04:09.520283    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/628849b1-f592-48a1-af55-d6f78351be63-samples-operator-tls podName:628849b1-f592-48a1-af55-d6f78351be63 nodeName:}" failed. No retries permitted until 2026-07-16 11:04:13.52027374 +0000 UTC m=+46.154495176 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/628849b1-f592-48a1-af55-d6f78351be63-samples-operator-tls") pod "cluster-samples-operator-65db858894-982zd" (UID: "628849b1-f592-48a1-af55-d6f78351be63") : secret "samples-operator-tls" not found
Jul 16 11:04:09.520312 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:04:09.520301    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ae4a146-99bf-422f-982f-0dab385b060b-cluster-monitoring-operator-tls podName:2ae4a146-99bf-422f-982f-0dab385b060b nodeName:}" failed. No retries permitted until 2026-07-16 11:04:13.520291744 +0000 UTC m=+46.154513179 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/2ae4a146-99bf-422f-982f-0dab385b060b-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-9ztqs" (UID: "2ae4a146-99bf-422f-982f-0dab385b060b") : secret "cluster-monitoring-operator-tls" not found
Jul 16 11:04:09.758613 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:09.758346    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qxpqt_c24aaddd-b44c-4155-a7c5-fadb281f9b39/konnectivity-agent/0.log" limit=1024
Jul 16 11:04:09.857196 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:09.857162    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qxpqt_c24aaddd-b44c-4155-a7c5-fadb281f9b39/konnectivity-agent/0.log" limit=1024
Jul 16 11:04:09.947081 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:09.946953    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qxpqt_c24aaddd-b44c-4155-a7c5-fadb281f9b39/konnectivity-agent/0.log" limit=1024
Jul 16 11:04:10.430120 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:10.430076    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9fceac27-6fd9-4091-af74-783b21616981-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-vgn2h\" (UID: \"9fceac27-6fd9-4091-af74-783b21616981\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-vgn2h"
Jul 16 11:04:10.430484 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:04:10.430284    2583 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 16 11:04:10.430484 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:04:10.430389    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9fceac27-6fd9-4091-af74-783b21616981-networking-console-plugin-cert podName:9fceac27-6fd9-4091-af74-783b21616981 nodeName:}" failed. No retries permitted until 2026-07-16 11:04:12.430364373 +0000 UTC m=+45.064585811 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/9fceac27-6fd9-4091-af74-783b21616981-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-vgn2h" (UID: "9fceac27-6fd9-4091-af74-783b21616981") : secret "networking-console-plugin-cert" not found
Jul 16 11:04:12.444086 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:12.444034    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9fceac27-6fd9-4091-af74-783b21616981-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-vgn2h\" (UID: \"9fceac27-6fd9-4091-af74-783b21616981\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-vgn2h"
Jul 16 11:04:12.444546 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:04:12.444200    2583 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 16 11:04:12.444546 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:04:12.444272    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9fceac27-6fd9-4091-af74-783b21616981-networking-console-plugin-cert podName:9fceac27-6fd9-4091-af74-783b21616981 nodeName:}" failed. No retries permitted until 2026-07-16 11:04:16.444253592 +0000 UTC m=+49.078475024 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/9fceac27-6fd9-4091-af74-783b21616981-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-vgn2h" (UID: "9fceac27-6fd9-4091-af74-783b21616981") : secret "networking-console-plugin-cert" not found
Jul 16 11:04:13.554920 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:13.554877    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2ae4a146-99bf-422f-982f-0dab385b060b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-9ztqs\" (UID: \"2ae4a146-99bf-422f-982f-0dab385b060b\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-9ztqs"
Jul 16 11:04:13.555354 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:13.554953    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/921e629b-d5a3-4bf6-8bad-0131d0a812ea-metrics-tls\") pod \"dns-default-5649t\" (UID: \"921e629b-d5a3-4bf6-8bad-0131d0a812ea\") " pod="openshift-dns/dns-default-5649t"
Jul 16 11:04:13.555354 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:13.555035    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ec3eb24d-8f3c-40b7-bfe9-a34fccdc20cd-service-ca-bundle\") pod \"router-default-76cb9bc7db-x2cnj\" (UID: \"ec3eb24d-8f3c-40b7-bfe9-a34fccdc20cd\") " pod="openshift-ingress/router-default-76cb9bc7db-x2cnj"
Jul 16 11:04:13.555354 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:04:13.555050    2583 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 16 11:04:13.555354 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:04:13.555097    2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 16 11:04:13.555354 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:04:13.555113    2583 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 16 11:04:13.555354 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:13.555057    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6a4b5dde-f664-4d97-889e-bfe4d4df1416-registry-tls\") pod \"image-registry-65f68df599-zw6jf\" (UID: \"6a4b5dde-f664-4d97-889e-bfe4d4df1416\") " pod="openshift-image-registry/image-registry-65f68df599-zw6jf"
Jul 16 11:04:13.555354 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:04:13.555124    2583 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-65f68df599-zw6jf: secret "image-registry-tls" not found
Jul 16 11:04:13.555354 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:04:13.555117    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ae4a146-99bf-422f-982f-0dab385b060b-cluster-monitoring-operator-tls podName:2ae4a146-99bf-422f-982f-0dab385b060b nodeName:}" failed. No retries permitted until 2026-07-16 11:04:21.555099478 +0000 UTC m=+54.189320913 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/2ae4a146-99bf-422f-982f-0dab385b060b-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-9ztqs" (UID: "2ae4a146-99bf-422f-982f-0dab385b060b") : secret "cluster-monitoring-operator-tls" not found
Jul 16 11:04:13.555354 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:04:13.555240    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/921e629b-d5a3-4bf6-8bad-0131d0a812ea-metrics-tls podName:921e629b-d5a3-4bf6-8bad-0131d0a812ea nodeName:}" failed. No retries permitted until 2026-07-16 11:04:21.55521319 +0000 UTC m=+54.189434638 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/921e629b-d5a3-4bf6-8bad-0131d0a812ea-metrics-tls") pod "dns-default-5649t" (UID: "921e629b-d5a3-4bf6-8bad-0131d0a812ea") : secret "dns-default-metrics-tls" not found
Jul 16 11:04:13.555354 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:04:13.555262    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ec3eb24d-8f3c-40b7-bfe9-a34fccdc20cd-service-ca-bundle podName:ec3eb24d-8f3c-40b7-bfe9-a34fccdc20cd nodeName:}" failed. No retries permitted until 2026-07-16 11:04:21.555250797 +0000 UTC m=+54.189472274 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/ec3eb24d-8f3c-40b7-bfe9-a34fccdc20cd-service-ca-bundle") pod "router-default-76cb9bc7db-x2cnj" (UID: "ec3eb24d-8f3c-40b7-bfe9-a34fccdc20cd") : configmap references non-existent config key: service-ca.crt
Jul 16 11:04:13.555354 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:04:13.555278    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6a4b5dde-f664-4d97-889e-bfe4d4df1416-registry-tls podName:6a4b5dde-f664-4d97-889e-bfe4d4df1416 nodeName:}" failed. No retries permitted until 2026-07-16 11:04:21.555270008 +0000 UTC m=+54.189491479 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/6a4b5dde-f664-4d97-889e-bfe4d4df1416-registry-tls") pod "image-registry-65f68df599-zw6jf" (UID: "6a4b5dde-f664-4d97-889e-bfe4d4df1416") : secret "image-registry-tls" not found
Jul 16 11:04:13.555354 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:13.555328    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ec3eb24d-8f3c-40b7-bfe9-a34fccdc20cd-metrics-certs\") pod \"router-default-76cb9bc7db-x2cnj\" (UID: \"ec3eb24d-8f3c-40b7-bfe9-a34fccdc20cd\") " pod="openshift-ingress/router-default-76cb9bc7db-x2cnj"
Jul 16 11:04:13.555953 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:13.555369    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/628849b1-f592-48a1-af55-d6f78351be63-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-982zd\" (UID: \"628849b1-f592-48a1-af55-d6f78351be63\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-982zd"
Jul 16 11:04:13.555953 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:04:13.555438    2583 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 16 11:04:13.555953 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:04:13.555472    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ec3eb24d-8f3c-40b7-bfe9-a34fccdc20cd-metrics-certs podName:ec3eb24d-8f3c-40b7-bfe9-a34fccdc20cd nodeName:}" failed. No retries permitted until 2026-07-16 11:04:21.555459848 +0000 UTC m=+54.189681280 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ec3eb24d-8f3c-40b7-bfe9-a34fccdc20cd-metrics-certs") pod "router-default-76cb9bc7db-x2cnj" (UID: "ec3eb24d-8f3c-40b7-bfe9-a34fccdc20cd") : secret "router-metrics-certs-default" not found
Jul 16 11:04:13.555953 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:04:13.555476    2583 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 16 11:04:13.555953 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:04:13.555530    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/628849b1-f592-48a1-af55-d6f78351be63-samples-operator-tls podName:628849b1-f592-48a1-af55-d6f78351be63 nodeName:}" failed. No retries permitted until 2026-07-16 11:04:21.555517617 +0000 UTC m=+54.189739065 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/628849b1-f592-48a1-af55-d6f78351be63-samples-operator-tls") pod "cluster-samples-operator-65db858894-982zd" (UID: "628849b1-f592-48a1-af55-d6f78351be63") : secret "samples-operator-tls" not found
Jul 16 11:04:15.635785 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:15.635745    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-4mhsn" event={"ID":"8dcffe9f-afd4-4e68-9c2a-a6d824df311e","Type":"ContainerStarted","Data":"3aa0109f66fe4f15fadbb8f1f75d544a5bb5a3dd84e4693c102046992172aeb1"}
Jul 16 11:04:15.637563 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:15.637535    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-85d88" event={"ID":"e1f38c54-19be-4f6f-872b-39093678a7ad","Type":"ContainerStarted","Data":"e1e94a4013b98dabc8afaf41665a18c0bda154aeb7fa382747e18afa75c8e0bd"}
Jul 16 11:04:15.641860 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:15.641835    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8r89z" event={"ID":"6fd1e802-78a8-4991-99ee-47cfea84ab62","Type":"ContainerStarted","Data":"ebe1c951cec687f52c5cca3a41b75a4e67fba01aba29e050dbc142e7148b3a12"}
Jul 16 11:04:15.643383 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:15.643361    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-7gl88" event={"ID":"543513a9-883b-4f90-8478-eb51b3186af1","Type":"ContainerStarted","Data":"cd09290e5f7521325b0cbd28367a0d41ea2da2fc67abc6b8a3582e4a96eee4bd"}
Jul 16 11:04:15.644879 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:15.644856    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-7d8g4" event={"ID":"369fdbca-23e1-4389-9b69-4f09463fc7bc","Type":"ContainerStarted","Data":"1560d3c17687d36bbf055163391d59100f9d864cded78f3b7faf6f51b9e43fdd"}
Jul 16 11:04:15.646541 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:15.646524    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-9lm5s_f64f15a3-5a9f-4737-bc4f-73d5ec311e41/console-operator/0.log"
Jul 16 11:04:15.646659 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:15.646561    2583 generic.go:356] "Generic (PLEG): container finished" podID="f64f15a3-5a9f-4737-bc4f-73d5ec311e41" containerID="38ad8fb224a6b2b6db7131da7b3be341d23cd578cb5c025899552bd08852e499" exitCode=255
Jul 16 11:04:15.646659 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:15.646591    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-9lm5s" event={"ID":"f64f15a3-5a9f-4737-bc4f-73d5ec311e41","Type":"ContainerDied","Data":"38ad8fb224a6b2b6db7131da7b3be341d23cd578cb5c025899552bd08852e499"}
Jul 16 11:04:15.647331 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:15.647312    2583 scope.go:117] "RemoveContainer" containerID="38ad8fb224a6b2b6db7131da7b3be341d23cd578cb5c025899552bd08852e499"
Jul 16 11:04:15.648751 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:15.648399    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-d496q" event={"ID":"ca7a1d19-213d-4ce9-8a34-4a93ddcab230","Type":"ContainerStarted","Data":"5043b65632f19fe9ba47e852e267d9f1c9dbfbd3ff389923880288438a748e7b"}
Jul 16 11:04:15.650555 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:15.650536    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-lx49b" event={"ID":"c839c027-e922-48f5-9940-d293b38aef05","Type":"ContainerStarted","Data":"2ab6722c18c456d276b495805cba05191a6de7f6554af801831f6d9b34620ede"}
Jul 16 11:04:15.653103 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:15.653047    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-4mhsn" podStartSLOduration=2.802625676 podStartE2EDuration="10.653029309s" totalImagesPullingTime="7.850403633s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 11:04:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 11:04:15.65231333 +0000 UTC m=+48.286534783" watchObservedRunningTime="2026-07-16 11:04:15.653029309 +0000 UTC m=+48.287250765"
Jul 16 11:04:15.675492 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:15.675139    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-85d88" podStartSLOduration=27.830192386 podStartE2EDuration="35.675114645s" totalImagesPullingTime="7.844922259s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 11:03:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 11:04:15.673284865 +0000 UTC m=+48.307506319" watchObservedRunningTime="2026-07-16 11:04:15.675114645 +0000 UTC m=+48.309336097"
Jul 16 11:04:15.719868 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:15.719568    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-8r89z" podStartSLOduration=13.77559543 podStartE2EDuration="45.719544853s" totalImagesPullingTime="31.943949423s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 11:03:30 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 11:04:15.718764201 +0000 UTC m=+48.352985681" watchObservedRunningTime="2026-07-16 11:04:15.719544853 +0000 UTC m=+48.353766309"
Jul 16 11:04:15.720953 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:15.720687    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-68d9dd94cf-lx49b" podStartSLOduration=27.875335651 podStartE2EDuration="35.720672976s" totalImagesPullingTime="7.845337325s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 11:03:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 11:04:15.694967729 +0000 UTC m=+48.329189184" watchObservedRunningTime="2026-07-16 11:04:15.720672976 +0000 UTC m=+48.354894432"
Jul 16 11:04:15.740289 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:15.739339    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-d496q" podStartSLOduration=27.928689709 podStartE2EDuration="35.739318048s" totalImagesPullingTime="7.810628339s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 11:03:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 11:04:15.737866681 +0000 UTC m=+48.372088138" watchObservedRunningTime="2026-07-16 11:04:15.739318048 +0000 UTC m=+48.373539503"
Jul 16 11:04:15.777990 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:15.777926    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-7d8g4" podStartSLOduration=27.922002119 podStartE2EDuration="35.777905881s" totalImagesPullingTime="7.855903762s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 11:03:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 11:04:15.775894341 +0000 UTC m=+48.410115796" watchObservedRunningTime="2026-07-16 11:04:15.777905881 +0000 UTC m=+48.412127337"
Jul 16 11:04:15.846625 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:15.845426    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-7gl88" podStartSLOduration=28.001656802 podStartE2EDuration="35.845393426s" totalImagesPullingTime="7.843736624s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 11:03:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 11:04:15.844769142 +0000 UTC m=+48.478990596" watchObservedRunningTime="2026-07-16 11:04:15.845393426 +0000 UTC m=+48.479614881"
Jul 16 11:04:16.107519 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:16.107480    2583 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7866497494-9lm5s"
Jul 16 11:04:16.107739 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:16.107547    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7866497494-9lm5s"
Jul 16 11:04:16.326437 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:16.326402    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-kqhbl"]
Jul 16 11:04:16.352226 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:16.352154    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-kqhbl"]
Jul 16 11:04:16.352510 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:16.352301    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-kqhbl"
Jul 16 11:04:16.354898 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:16.354763    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 16 11:04:16.354898 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:16.354763    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 16 11:04:16.354898 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:16.354767    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-lhr85\""
Jul 16 11:04:16.383232 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:16.383195    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/19a1fad5-ffe8-43e3-8c58-028cf08b6836-crio-socket\") pod \"insights-runtime-extractor-kqhbl\" (UID: \"19a1fad5-ffe8-43e3-8c58-028cf08b6836\") " pod="openshift-insights/insights-runtime-extractor-kqhbl"
Jul 16 11:04:16.383529 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:16.383510    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/19a1fad5-ffe8-43e3-8c58-028cf08b6836-data-volume\") pod \"insights-runtime-extractor-kqhbl\" (UID: \"19a1fad5-ffe8-43e3-8c58-028cf08b6836\") " pod="openshift-insights/insights-runtime-extractor-kqhbl"
Jul 16 11:04:16.383721 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:16.383704    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/19a1fad5-ffe8-43e3-8c58-028cf08b6836-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kqhbl\" (UID: \"19a1fad5-ffe8-43e3-8c58-028cf08b6836\") " pod="openshift-insights/insights-runtime-extractor-kqhbl"
Jul 16 11:04:16.383869 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:16.383854    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/19a1fad5-ffe8-43e3-8c58-028cf08b6836-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kqhbl\" (UID: \"19a1fad5-ffe8-43e3-8c58-028cf08b6836\") " pod="openshift-insights/insights-runtime-extractor-kqhbl"
Jul 16 11:04:16.384071 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:16.384058    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-msl4l\" (UniqueName: \"kubernetes.io/projected/19a1fad5-ffe8-43e3-8c58-028cf08b6836-kube-api-access-msl4l\") pod \"insights-runtime-extractor-kqhbl\" (UID: \"19a1fad5-ffe8-43e3-8c58-028cf08b6836\") " pod="openshift-insights/insights-runtime-extractor-kqhbl"
Jul 16 11:04:16.485001 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:16.484970    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9fceac27-6fd9-4091-af74-783b21616981-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-vgn2h\" (UID: \"9fceac27-6fd9-4091-af74-783b21616981\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-vgn2h"
Jul 16 11:04:16.485001 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:16.485007    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-msl4l\" (UniqueName: \"kubernetes.io/projected/19a1fad5-ffe8-43e3-8c58-028cf08b6836-kube-api-access-msl4l\") pod \"insights-runtime-extractor-kqhbl\" (UID: \"19a1fad5-ffe8-43e3-8c58-028cf08b6836\") " pod="openshift-insights/insights-runtime-extractor-kqhbl"
Jul 16 11:04:16.485214 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:16.485129    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/19a1fad5-ffe8-43e3-8c58-028cf08b6836-crio-socket\") pod \"insights-runtime-extractor-kqhbl\" (UID: \"19a1fad5-ffe8-43e3-8c58-028cf08b6836\") " pod="openshift-insights/insights-runtime-extractor-kqhbl"
Jul 16 11:04:16.485214 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:16.485164    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/19a1fad5-ffe8-43e3-8c58-028cf08b6836-data-volume\") pod \"insights-runtime-extractor-kqhbl\" (UID: \"19a1fad5-ffe8-43e3-8c58-028cf08b6836\") " pod="openshift-insights/insights-runtime-extractor-kqhbl"
Jul 16 11:04:16.485278 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:04:16.485251    2583 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 16 11:04:16.485320 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:04:16.485308    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9fceac27-6fd9-4091-af74-783b21616981-networking-console-plugin-cert podName:9fceac27-6fd9-4091-af74-783b21616981 nodeName:}" failed. No retries permitted until 2026-07-16 11:04:24.485293269 +0000 UTC m=+57.119514705 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/9fceac27-6fd9-4091-af74-783b21616981-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-vgn2h" (UID: "9fceac27-6fd9-4091-af74-783b21616981") : secret "networking-console-plugin-cert" not found
Jul 16 11:04:16.485378 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:16.485326    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/19a1fad5-ffe8-43e3-8c58-028cf08b6836-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kqhbl\" (UID: \"19a1fad5-ffe8-43e3-8c58-028cf08b6836\") " pod="openshift-insights/insights-runtime-extractor-kqhbl"
Jul 16 11:04:16.485378 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:16.485349    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/19a1fad5-ffe8-43e3-8c58-028cf08b6836-crio-socket\") pod \"insights-runtime-extractor-kqhbl\" (UID: \"19a1fad5-ffe8-43e3-8c58-028cf08b6836\") " pod="openshift-insights/insights-runtime-extractor-kqhbl"
Jul 16 11:04:16.485378 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:16.485357    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/19a1fad5-ffe8-43e3-8c58-028cf08b6836-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kqhbl\" (UID: \"19a1fad5-ffe8-43e3-8c58-028cf08b6836\") " pod="openshift-insights/insights-runtime-extractor-kqhbl"
Jul 16 11:04:16.485519 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:04:16.485393    2583 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 16 11:04:16.485519 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:04:16.485424    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/19a1fad5-ffe8-43e3-8c58-028cf08b6836-kube-rbac-proxy-cm podName:19a1fad5-ffe8-43e3-8c58-028cf08b6836 nodeName:}" failed. No retries permitted until 2026-07-16 11:04:16.985413905 +0000 UTC m=+49.619635337 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/19a1fad5-ffe8-43e3-8c58-028cf08b6836-kube-rbac-proxy-cm") pod "insights-runtime-extractor-kqhbl" (UID: "19a1fad5-ffe8-43e3-8c58-028cf08b6836") : configmap "kube-rbac-proxy" not found
Jul 16 11:04:16.485519 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:04:16.485441    2583 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 16 11:04:16.485519 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:16.485472    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/19a1fad5-ffe8-43e3-8c58-028cf08b6836-data-volume\") pod \"insights-runtime-extractor-kqhbl\" (UID: \"19a1fad5-ffe8-43e3-8c58-028cf08b6836\") " pod="openshift-insights/insights-runtime-extractor-kqhbl"
Jul 16 11:04:16.485519 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:04:16.485496    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/19a1fad5-ffe8-43e3-8c58-028cf08b6836-insights-runtime-extractor-tls podName:19a1fad5-ffe8-43e3-8c58-028cf08b6836 nodeName:}" failed. No retries permitted until 2026-07-16 11:04:16.985480082 +0000 UTC m=+49.619701527 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/19a1fad5-ffe8-43e3-8c58-028cf08b6836-insights-runtime-extractor-tls") pod "insights-runtime-extractor-kqhbl" (UID: "19a1fad5-ffe8-43e3-8c58-028cf08b6836") : secret "insights-runtime-extractor-tls" not found
Jul 16 11:04:16.495226 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:16.495197    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-msl4l\" (UniqueName: \"kubernetes.io/projected/19a1fad5-ffe8-43e3-8c58-028cf08b6836-kube-api-access-msl4l\") pod \"insights-runtime-extractor-kqhbl\" (UID: \"19a1fad5-ffe8-43e3-8c58-028cf08b6836\") " pod="openshift-insights/insights-runtime-extractor-kqhbl"
Jul 16 11:04:16.656030 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:16.656002    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-9lm5s_f64f15a3-5a9f-4737-bc4f-73d5ec311e41/console-operator/1.log"
Jul 16 11:04:16.656440 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:16.656425    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-9lm5s_f64f15a3-5a9f-4737-bc4f-73d5ec311e41/console-operator/0.log"
Jul 16 11:04:16.656496 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:16.656457    2583 generic.go:356] "Generic (PLEG): container finished" podID="f64f15a3-5a9f-4737-bc4f-73d5ec311e41" containerID="9fd623aecb8191ec1e80c9c24f5250c1efb8706153165961f9555737a510e1d4" exitCode=255
Jul 16 11:04:16.656551 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:16.656535    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-9lm5s" event={"ID":"f64f15a3-5a9f-4737-bc4f-73d5ec311e41","Type":"ContainerDied","Data":"9fd623aecb8191ec1e80c9c24f5250c1efb8706153165961f9555737a510e1d4"}
Jul 16 11:04:16.656621 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:16.656586    2583 scope.go:117] "RemoveContainer" containerID="38ad8fb224a6b2b6db7131da7b3be341d23cd578cb5c025899552bd08852e499"
Jul 16 11:04:16.656859 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:16.656829    2583 scope.go:117] "RemoveContainer" containerID="9fd623aecb8191ec1e80c9c24f5250c1efb8706153165961f9555737a510e1d4"
Jul 16 11:04:16.657066 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:04:16.657033    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-9lm5s_openshift-console-operator(f64f15a3-5a9f-4737-bc4f-73d5ec311e41)\"" pod="openshift-console-operator/console-operator-7866497494-9lm5s" podUID="f64f15a3-5a9f-4737-bc4f-73d5ec311e41"
Jul 16 11:04:16.991301 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:16.991266    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/19a1fad5-ffe8-43e3-8c58-028cf08b6836-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kqhbl\" (UID: \"19a1fad5-ffe8-43e3-8c58-028cf08b6836\") " pod="openshift-insights/insights-runtime-extractor-kqhbl"
Jul 16 11:04:16.991470 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:16.991321    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/19a1fad5-ffe8-43e3-8c58-028cf08b6836-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kqhbl\" (UID: \"19a1fad5-ffe8-43e3-8c58-028cf08b6836\") " pod="openshift-insights/insights-runtime-extractor-kqhbl"
Jul 16 11:04:16.991470 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:04:16.991421    2583 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 16 11:04:16.991470 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:04:16.991451    2583 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 16 11:04:16.991582 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:04:16.991486    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/19a1fad5-ffe8-43e3-8c58-028cf08b6836-insights-runtime-extractor-tls podName:19a1fad5-ffe8-43e3-8c58-028cf08b6836 nodeName:}" failed. No retries permitted until 2026-07-16 11:04:17.991468681 +0000 UTC m=+50.625690114 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/19a1fad5-ffe8-43e3-8c58-028cf08b6836-insights-runtime-extractor-tls") pod "insights-runtime-extractor-kqhbl" (UID: "19a1fad5-ffe8-43e3-8c58-028cf08b6836") : secret "insights-runtime-extractor-tls" not found
Jul 16 11:04:16.991582 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:04:16.991550    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/19a1fad5-ffe8-43e3-8c58-028cf08b6836-kube-rbac-proxy-cm podName:19a1fad5-ffe8-43e3-8c58-028cf08b6836 nodeName:}" failed. No retries permitted until 2026-07-16 11:04:17.991541147 +0000 UTC m=+50.625762597 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/19a1fad5-ffe8-43e3-8c58-028cf08b6836-kube-rbac-proxy-cm") pod "insights-runtime-extractor-kqhbl" (UID: "19a1fad5-ffe8-43e3-8c58-028cf08b6836") : configmap "kube-rbac-proxy" not found
Jul 16 11:04:17.253655 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:17.253529    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-l5xq6"]
Jul 16 11:04:17.286992 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:17.286953    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-l5xq6"]
Jul 16 11:04:17.287149 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:17.287059    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-l5xq6"
Jul 16 11:04:17.289728 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:17.289697    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Jul 16 11:04:17.289851 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:17.289754    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Jul 16 11:04:17.290568 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:17.290552    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-hpfgm\""
Jul 16 11:04:17.294291 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:17.294210    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xw96q\" (UniqueName: \"kubernetes.io/projected/a5de1249-3e28-4a28-a94a-eee038063ea0-kube-api-access-xw96q\") pod \"migrator-7f59b54f6c-l5xq6\" (UID: \"a5de1249-3e28-4a28-a94a-eee038063ea0\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-l5xq6"
Jul 16 11:04:17.395161 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:17.395123    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xw96q\" (UniqueName: \"kubernetes.io/projected/a5de1249-3e28-4a28-a94a-eee038063ea0-kube-api-access-xw96q\") pod \"migrator-7f59b54f6c-l5xq6\" (UID: \"a5de1249-3e28-4a28-a94a-eee038063ea0\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-l5xq6"
Jul 16 11:04:17.410138 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:17.410102    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xw96q\" (UniqueName: \"kubernetes.io/projected/a5de1249-3e28-4a28-a94a-eee038063ea0-kube-api-access-xw96q\") pod \"migrator-7f59b54f6c-l5xq6\" (UID: \"a5de1249-3e28-4a28-a94a-eee038063ea0\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-l5xq6"
Jul 16 11:04:17.597116 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:17.597037    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-l5xq6"
Jul 16 11:04:17.663971 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:17.663938    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-9lm5s_f64f15a3-5a9f-4737-bc4f-73d5ec311e41/console-operator/1.log"
Jul 16 11:04:17.664356 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:17.664338    2583 scope.go:117] "RemoveContainer" containerID="9fd623aecb8191ec1e80c9c24f5250c1efb8706153165961f9555737a510e1d4"
Jul 16 11:04:17.664561 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:04:17.664537    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-9lm5s_openshift-console-operator(f64f15a3-5a9f-4737-bc4f-73d5ec311e41)\"" pod="openshift-console-operator/console-operator-7866497494-9lm5s" podUID="f64f15a3-5a9f-4737-bc4f-73d5ec311e41"
Jul 16 11:04:17.724305 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:17.724279    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-z5dtv_a3d44785-92d8-4a5e-bc2e-b91f4fa8b509/dns-node-resolver/0.log"
Jul 16 11:04:17.742484 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:17.742453    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-l5xq6"]
Jul 16 11:04:17.745637 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:04:17.745589    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5de1249_3e28_4a28_a94a_eee038063ea0.slice/crio-a85c78d28cc835aebdb52b56a48af95cebe8ed10adc715b515798ef23ccf4afc WatchSource:0}: Error finding container a85c78d28cc835aebdb52b56a48af95cebe8ed10adc715b515798ef23ccf4afc: Status 404 returned error can't find the container with id a85c78d28cc835aebdb52b56a48af95cebe8ed10adc715b515798ef23ccf4afc
Jul 16 11:04:18.001836 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:18.001798    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/19a1fad5-ffe8-43e3-8c58-028cf08b6836-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kqhbl\" (UID: \"19a1fad5-ffe8-43e3-8c58-028cf08b6836\") " pod="openshift-insights/insights-runtime-extractor-kqhbl"
Jul 16 11:04:18.002063 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:04:18.001938    2583 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 16 11:04:18.002063 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:18.001973    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/19a1fad5-ffe8-43e3-8c58-028cf08b6836-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kqhbl\" (UID: \"19a1fad5-ffe8-43e3-8c58-028cf08b6836\") " pod="openshift-insights/insights-runtime-extractor-kqhbl"
Jul 16 11:04:18.002063 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:04:18.002004    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/19a1fad5-ffe8-43e3-8c58-028cf08b6836-kube-rbac-proxy-cm podName:19a1fad5-ffe8-43e3-8c58-028cf08b6836 nodeName:}" failed. No retries permitted until 2026-07-16 11:04:20.001987036 +0000 UTC m=+52.636208469 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/19a1fad5-ffe8-43e3-8c58-028cf08b6836-kube-rbac-proxy-cm") pod "insights-runtime-extractor-kqhbl" (UID: "19a1fad5-ffe8-43e3-8c58-028cf08b6836") : configmap "kube-rbac-proxy" not found
Jul 16 11:04:18.002224 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:04:18.002072    2583 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 16 11:04:18.002224 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:04:18.002117    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/19a1fad5-ffe8-43e3-8c58-028cf08b6836-insights-runtime-extractor-tls podName:19a1fad5-ffe8-43e3-8c58-028cf08b6836 nodeName:}" failed. No retries permitted until 2026-07-16 11:04:20.002106195 +0000 UTC m=+52.636327628 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/19a1fad5-ffe8-43e3-8c58-028cf08b6836-insights-runtime-extractor-tls") pod "insights-runtime-extractor-kqhbl" (UID: "19a1fad5-ffe8-43e3-8c58-028cf08b6836") : secret "insights-runtime-extractor-tls" not found
Jul 16 11:04:18.322995 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:18.322923    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-6xfs6_0244f4b4-dd39-4870-97be-8378a99c93a2/node-ca/0.log"
Jul 16 11:04:18.608495 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:18.608411    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-csw7v"]
Jul 16 11:04:18.627699 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:18.627662    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-csw7v"]
Jul 16 11:04:18.627867 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:18.627821    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-79f68b9f4f-csw7v"
Jul 16 11:04:18.630634 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:18.630584    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-nf5ll\""
Jul 16 11:04:18.630745 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:18.630639    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Jul 16 11:04:18.630818 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:18.630799    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Jul 16 11:04:18.631566 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:18.631536    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Jul 16 11:04:18.631566 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:18.631549    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Jul 16 11:04:18.668653 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:18.668623    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-l5xq6" event={"ID":"a5de1249-3e28-4a28-a94a-eee038063ea0","Type":"ContainerStarted","Data":"a85c78d28cc835aebdb52b56a48af95cebe8ed10adc715b515798ef23ccf4afc"}
Jul 16 11:04:18.708316 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:18.708278    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/c5f2b707-27d2-4347-99cb-5b68984777fc-signing-cabundle\") pod \"service-ca-79f68b9f4f-csw7v\" (UID: \"c5f2b707-27d2-4347-99cb-5b68984777fc\") " pod="openshift-service-ca/service-ca-79f68b9f4f-csw7v"
Jul 16 11:04:18.708503 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:18.708367    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w6sjl\" (UniqueName: \"kubernetes.io/projected/c5f2b707-27d2-4347-99cb-5b68984777fc-kube-api-access-w6sjl\") pod \"service-ca-79f68b9f4f-csw7v\" (UID: \"c5f2b707-27d2-4347-99cb-5b68984777fc\") " pod="openshift-service-ca/service-ca-79f68b9f4f-csw7v"
Jul 16 11:04:18.708503 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:18.708420    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/c5f2b707-27d2-4347-99cb-5b68984777fc-signing-key\") pod \"service-ca-79f68b9f4f-csw7v\" (UID: \"c5f2b707-27d2-4347-99cb-5b68984777fc\") " pod="openshift-service-ca/service-ca-79f68b9f4f-csw7v"
Jul 16 11:04:18.809782 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:18.809739    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/c5f2b707-27d2-4347-99cb-5b68984777fc-signing-cabundle\") pod \"service-ca-79f68b9f4f-csw7v\" (UID: \"c5f2b707-27d2-4347-99cb-5b68984777fc\") " pod="openshift-service-ca/service-ca-79f68b9f4f-csw7v"
Jul 16 11:04:18.809957 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:18.809853    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w6sjl\" (UniqueName: \"kubernetes.io/projected/c5f2b707-27d2-4347-99cb-5b68984777fc-kube-api-access-w6sjl\") pod \"service-ca-79f68b9f4f-csw7v\" (UID: \"c5f2b707-27d2-4347-99cb-5b68984777fc\") " pod="openshift-service-ca/service-ca-79f68b9f4f-csw7v"
Jul 16 11:04:18.809957 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:18.809880    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/c5f2b707-27d2-4347-99cb-5b68984777fc-signing-key\") pod \"service-ca-79f68b9f4f-csw7v\" (UID: \"c5f2b707-27d2-4347-99cb-5b68984777fc\") " pod="openshift-service-ca/service-ca-79f68b9f4f-csw7v"
Jul 16 11:04:18.810420 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:18.810396    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/c5f2b707-27d2-4347-99cb-5b68984777fc-signing-cabundle\") pod \"service-ca-79f68b9f4f-csw7v\" (UID: \"c5f2b707-27d2-4347-99cb-5b68984777fc\") " pod="openshift-service-ca/service-ca-79f68b9f4f-csw7v"
Jul 16 11:04:18.812346 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:18.812320    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/c5f2b707-27d2-4347-99cb-5b68984777fc-signing-key\") pod \"service-ca-79f68b9f4f-csw7v\" (UID: \"c5f2b707-27d2-4347-99cb-5b68984777fc\") " pod="openshift-service-ca/service-ca-79f68b9f4f-csw7v"
Jul 16 11:04:18.819846 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:18.819822    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w6sjl\" (UniqueName: \"kubernetes.io/projected/c5f2b707-27d2-4347-99cb-5b68984777fc-kube-api-access-w6sjl\") pod \"service-ca-79f68b9f4f-csw7v\" (UID: \"c5f2b707-27d2-4347-99cb-5b68984777fc\") " pod="openshift-service-ca/service-ca-79f68b9f4f-csw7v"
Jul 16 11:04:18.939356 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:18.939320    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-79f68b9f4f-csw7v"
Jul 16 11:04:19.122398 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:19.122152    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-4mhsn_8dcffe9f-afd4-4e68-9c2a-a6d824df311e/serve-healthcheck-canary/0.log"
Jul 16 11:04:19.207406 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:19.207369    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-csw7v"]
Jul 16 11:04:19.210811 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:04:19.210786    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc5f2b707_27d2_4347_99cb_5b68984777fc.slice/crio-28f281386dfdd2842ec9399ccb2e7d034523cba580faca5fb0d0c2b2ccd6359e WatchSource:0}: Error finding container 28f281386dfdd2842ec9399ccb2e7d034523cba580faca5fb0d0c2b2ccd6359e: Status 404 returned error can't find the container with id 28f281386dfdd2842ec9399ccb2e7d034523cba580faca5fb0d0c2b2ccd6359e
Jul 16 11:04:19.674350 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:19.674310    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-79f68b9f4f-csw7v" event={"ID":"c5f2b707-27d2-4347-99cb-5b68984777fc","Type":"ContainerStarted","Data":"536e9c9f5cde7afca87316bedee2afc0c1ad4e04a2d47c998fd1da0ff6bfb641"}
Jul 16 11:04:19.674866 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:19.674366    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-79f68b9f4f-csw7v" event={"ID":"c5f2b707-27d2-4347-99cb-5b68984777fc","Type":"ContainerStarted","Data":"28f281386dfdd2842ec9399ccb2e7d034523cba580faca5fb0d0c2b2ccd6359e"}
Jul 16 11:04:19.676276 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:19.676250    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-l5xq6" event={"ID":"a5de1249-3e28-4a28-a94a-eee038063ea0","Type":"ContainerStarted","Data":"981ddbb79f83917ee2cb8c95f0cbdd4f392d31cb5c33315c32a91d2b59dbb4a3"}
Jul 16 11:04:19.676397 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:19.676298    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-l5xq6" event={"ID":"a5de1249-3e28-4a28-a94a-eee038063ea0","Type":"ContainerStarted","Data":"eb23bef432d48993e2e824b20b0b7d7344727e1d861220f50acec5cc7808fe3d"}
Jul 16 11:04:19.698929 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:19.698877    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-79f68b9f4f-csw7v" podStartSLOduration=1.698860968 podStartE2EDuration="1.698860968s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 11:04:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 11:04:19.697438162 +0000 UTC m=+52.331659648" watchObservedRunningTime="2026-07-16 11:04:19.698860968 +0000 UTC m=+52.333082466"
Jul 16 11:04:19.718517 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:19.718452    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-l5xq6" podStartSLOduration=1.385189142 podStartE2EDuration="2.718428387s" totalImagesPullingTime="1.333239245s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 11:04:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 11:04:19.717897388 +0000 UTC m=+52.352118845" watchObservedRunningTime="2026-07-16 11:04:19.718428387 +0000 UTC m=+52.352649847"
Jul 16 11:04:20.022881 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:20.022779    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/19a1fad5-ffe8-43e3-8c58-028cf08b6836-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kqhbl\" (UID: \"19a1fad5-ffe8-43e3-8c58-028cf08b6836\") " pod="openshift-insights/insights-runtime-extractor-kqhbl"
Jul 16 11:04:20.022881 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:20.022836    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/19a1fad5-ffe8-43e3-8c58-028cf08b6836-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kqhbl\" (UID: \"19a1fad5-ffe8-43e3-8c58-028cf08b6836\") " pod="openshift-insights/insights-runtime-extractor-kqhbl"
Jul 16 11:04:20.023098 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:04:20.022946    2583 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 16 11:04:20.023098 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:04:20.022977    2583 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 16 11:04:20.023098 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:04:20.023027    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/19a1fad5-ffe8-43e3-8c58-028cf08b6836-insights-runtime-extractor-tls podName:19a1fad5-ffe8-43e3-8c58-028cf08b6836 nodeName:}" failed. No retries permitted until 2026-07-16 11:04:24.023006813 +0000 UTC m=+56.657228250 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/19a1fad5-ffe8-43e3-8c58-028cf08b6836-insights-runtime-extractor-tls") pod "insights-runtime-extractor-kqhbl" (UID: "19a1fad5-ffe8-43e3-8c58-028cf08b6836") : secret "insights-runtime-extractor-tls" not found
Jul 16 11:04:20.023098 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:04:20.023045    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/19a1fad5-ffe8-43e3-8c58-028cf08b6836-kube-rbac-proxy-cm podName:19a1fad5-ffe8-43e3-8c58-028cf08b6836 nodeName:}" failed. No retries permitted until 2026-07-16 11:04:24.023036476 +0000 UTC m=+56.657257934 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/19a1fad5-ffe8-43e3-8c58-028cf08b6836-kube-rbac-proxy-cm") pod "insights-runtime-extractor-kqhbl" (UID: "19a1fad5-ffe8-43e3-8c58-028cf08b6836") : configmap "kube-rbac-proxy" not found
Jul 16 11:04:21.638963 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:21.638907    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/628849b1-f592-48a1-af55-d6f78351be63-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-982zd\" (UID: \"628849b1-f592-48a1-af55-d6f78351be63\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-982zd"
Jul 16 11:04:21.639558 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:21.639037    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2ae4a146-99bf-422f-982f-0dab385b060b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-9ztqs\" (UID: \"2ae4a146-99bf-422f-982f-0dab385b060b\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-9ztqs"
Jul 16 11:04:21.639558 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:21.639093    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/921e629b-d5a3-4bf6-8bad-0131d0a812ea-metrics-tls\") pod \"dns-default-5649t\" (UID: \"921e629b-d5a3-4bf6-8bad-0131d0a812ea\") " pod="openshift-dns/dns-default-5649t"
Jul 16 11:04:21.639558 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:21.639166    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ec3eb24d-8f3c-40b7-bfe9-a34fccdc20cd-service-ca-bundle\") pod \"router-default-76cb9bc7db-x2cnj\" (UID: \"ec3eb24d-8f3c-40b7-bfe9-a34fccdc20cd\") " pod="openshift-ingress/router-default-76cb9bc7db-x2cnj"
Jul 16 11:04:21.639558 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:21.639195    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6a4b5dde-f664-4d97-889e-bfe4d4df1416-registry-tls\") pod \"image-registry-65f68df599-zw6jf\" (UID: \"6a4b5dde-f664-4d97-889e-bfe4d4df1416\") " pod="openshift-image-registry/image-registry-65f68df599-zw6jf"
Jul 16 11:04:21.639558 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:21.639224    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ec3eb24d-8f3c-40b7-bfe9-a34fccdc20cd-metrics-certs\") pod \"router-default-76cb9bc7db-x2cnj\" (UID: \"ec3eb24d-8f3c-40b7-bfe9-a34fccdc20cd\") " pod="openshift-ingress/router-default-76cb9bc7db-x2cnj"
Jul 16 11:04:21.639558 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:04:21.639223    2583 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 16 11:04:21.639558 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:04:21.639298    2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 16 11:04:21.639558 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:04:21.639312    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ae4a146-99bf-422f-982f-0dab385b060b-cluster-monitoring-operator-tls podName:2ae4a146-99bf-422f-982f-0dab385b060b nodeName:}" failed. No retries permitted until 2026-07-16 11:04:37.639290072 +0000 UTC m=+70.273511524 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/2ae4a146-99bf-422f-982f-0dab385b060b-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-9ztqs" (UID: "2ae4a146-99bf-422f-982f-0dab385b060b") : secret "cluster-monitoring-operator-tls" not found
Jul 16 11:04:21.639558 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:04:21.639326    2583 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 16 11:04:21.639558 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:04:21.639370    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/921e629b-d5a3-4bf6-8bad-0131d0a812ea-metrics-tls podName:921e629b-d5a3-4bf6-8bad-0131d0a812ea nodeName:}" failed. No retries permitted until 2026-07-16 11:04:37.639352589 +0000 UTC m=+70.273574028 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/921e629b-d5a3-4bf6-8bad-0131d0a812ea-metrics-tls") pod "dns-default-5649t" (UID: "921e629b-d5a3-4bf6-8bad-0131d0a812ea") : secret "dns-default-metrics-tls" not found
Jul 16 11:04:21.639558 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:04:21.639390    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ec3eb24d-8f3c-40b7-bfe9-a34fccdc20cd-metrics-certs podName:ec3eb24d-8f3c-40b7-bfe9-a34fccdc20cd nodeName:}" failed. No retries permitted until 2026-07-16 11:04:37.639380468 +0000 UTC m=+70.273601916 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ec3eb24d-8f3c-40b7-bfe9-a34fccdc20cd-metrics-certs") pod "router-default-76cb9bc7db-x2cnj" (UID: "ec3eb24d-8f3c-40b7-bfe9-a34fccdc20cd") : secret "router-metrics-certs-default" not found
Jul 16 11:04:21.639558 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:04:21.639427    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ec3eb24d-8f3c-40b7-bfe9-a34fccdc20cd-service-ca-bundle podName:ec3eb24d-8f3c-40b7-bfe9-a34fccdc20cd nodeName:}" failed. No retries permitted until 2026-07-16 11:04:37.639413751 +0000 UTC m=+70.273635189 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/ec3eb24d-8f3c-40b7-bfe9-a34fccdc20cd-service-ca-bundle") pod "router-default-76cb9bc7db-x2cnj" (UID: "ec3eb24d-8f3c-40b7-bfe9-a34fccdc20cd") : configmap references non-existent config key: service-ca.crt
Jul 16 11:04:21.639558 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:04:21.639458    2583 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 16 11:04:21.639558 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:04:21.639470    2583 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-65f68df599-zw6jf: secret "image-registry-tls" not found
Jul 16 11:04:21.639558 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:04:21.639504    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6a4b5dde-f664-4d97-889e-bfe4d4df1416-registry-tls podName:6a4b5dde-f664-4d97-889e-bfe4d4df1416 nodeName:}" failed. No retries permitted until 2026-07-16 11:04:37.639495018 +0000 UTC m=+70.273716453 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/6a4b5dde-f664-4d97-889e-bfe4d4df1416-registry-tls") pod "image-registry-65f68df599-zw6jf" (UID: "6a4b5dde-f664-4d97-889e-bfe4d4df1416") : secret "image-registry-tls" not found
Jul 16 11:04:21.641719 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:21.641689    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/628849b1-f592-48a1-af55-d6f78351be63-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-982zd\" (UID: \"628849b1-f592-48a1-af55-d6f78351be63\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-982zd"
Jul 16 11:04:21.699823 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:21.699783    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-982zd"
Jul 16 11:04:21.854794 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:21.854644    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-982zd"]
Jul 16 11:04:22.694413 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:22.694372    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-982zd" event={"ID":"628849b1-f592-48a1-af55-d6f78351be63","Type":"ContainerStarted","Data":"aca8c7c6495f1129c0f0521118f28bdfe028c6f630ddd340718e6adab9779699"}
Jul 16 11:04:24.065617 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:24.065568    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/19a1fad5-ffe8-43e3-8c58-028cf08b6836-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kqhbl\" (UID: \"19a1fad5-ffe8-43e3-8c58-028cf08b6836\") " pod="openshift-insights/insights-runtime-extractor-kqhbl"
Jul 16 11:04:24.065941 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:24.065647    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/19a1fad5-ffe8-43e3-8c58-028cf08b6836-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kqhbl\" (UID: \"19a1fad5-ffe8-43e3-8c58-028cf08b6836\") " pod="openshift-insights/insights-runtime-extractor-kqhbl"
Jul 16 11:04:24.065941 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:04:24.065719    2583 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 16 11:04:24.065941 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:04:24.065789    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/19a1fad5-ffe8-43e3-8c58-028cf08b6836-insights-runtime-extractor-tls podName:19a1fad5-ffe8-43e3-8c58-028cf08b6836 nodeName:}" failed. No retries permitted until 2026-07-16 11:04:32.06577122 +0000 UTC m=+64.699992674 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/19a1fad5-ffe8-43e3-8c58-028cf08b6836-insights-runtime-extractor-tls") pod "insights-runtime-extractor-kqhbl" (UID: "19a1fad5-ffe8-43e3-8c58-028cf08b6836") : secret "insights-runtime-extractor-tls" not found
Jul 16 11:04:24.066157 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:24.066140    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/19a1fad5-ffe8-43e3-8c58-028cf08b6836-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kqhbl\" (UID: \"19a1fad5-ffe8-43e3-8c58-028cf08b6836\") " pod="openshift-insights/insights-runtime-extractor-kqhbl"
Jul 16 11:04:24.570363 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:24.570325    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9fceac27-6fd9-4091-af74-783b21616981-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-vgn2h\" (UID: \"9fceac27-6fd9-4091-af74-783b21616981\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-vgn2h"
Jul 16 11:04:24.570626 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:04:24.570498    2583 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 16 11:04:24.570626 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:04:24.570582    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9fceac27-6fd9-4091-af74-783b21616981-networking-console-plugin-cert podName:9fceac27-6fd9-4091-af74-783b21616981 nodeName:}" failed. No retries permitted until 2026-07-16 11:04:40.570562088 +0000 UTC m=+73.204783537 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/9fceac27-6fd9-4091-af74-783b21616981-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-vgn2h" (UID: "9fceac27-6fd9-4091-af74-783b21616981") : secret "networking-console-plugin-cert" not found
Jul 16 11:04:24.702013 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:24.701977    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-982zd" event={"ID":"628849b1-f592-48a1-af55-d6f78351be63","Type":"ContainerStarted","Data":"31aa3262301a7ec4c9e436cce6d124391d47f7666670020a16a3e4873144966f"}
Jul 16 11:04:24.702183 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:24.702030    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-982zd" event={"ID":"628849b1-f592-48a1-af55-d6f78351be63","Type":"ContainerStarted","Data":"42aa24411fc97cb2d686abbde87d1e359a0d5ae35c08ba3bb9c93c4fcf66993c"}
Jul 16 11:04:24.724373 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:24.724312    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-982zd" podStartSLOduration=42.775680258 podStartE2EDuration="44.724289411s" totalImagesPullingTime="1.948609153s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 11:03:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 11:04:24.722215784 +0000 UTC m=+57.356437266" watchObservedRunningTime="2026-07-16 11:04:24.724289411 +0000 UTC m=+57.358510868"
Jul 16 11:04:26.107973 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:26.107933    2583 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7866497494-9lm5s"
Jul 16 11:04:26.108449 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:26.107999    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7866497494-9lm5s"
Jul 16 11:04:26.108516 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:26.108463    2583 scope.go:117] "RemoveContainer" containerID="9fd623aecb8191ec1e80c9c24f5250c1efb8706153165961f9555737a510e1d4"
Jul 16 11:04:26.712385 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:26.712355    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-9lm5s_f64f15a3-5a9f-4737-bc4f-73d5ec311e41/console-operator/1.log"
Jul 16 11:04:26.712553 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:26.712422    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-9lm5s" event={"ID":"f64f15a3-5a9f-4737-bc4f-73d5ec311e41","Type":"ContainerStarted","Data":"523092b407a8462f76ad4efd51520d8a351cb9a6634586750b42a0dbb5023ece"}
Jul 16 11:04:26.712776 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:26.712748    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7866497494-9lm5s"
Jul 16 11:04:26.717943 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:26.717917    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-7866497494-9lm5s"
Jul 16 11:04:26.731325 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:26.731267    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-7866497494-9lm5s" podStartSLOduration=38.885854226 podStartE2EDuration="46.731246701s" totalImagesPullingTime="7.845392475s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 11:03:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 11:04:26.730937572 +0000 UTC m=+59.365159028" watchObservedRunningTime="2026-07-16 11:04:26.731246701 +0000 UTC m=+59.365468156"
Jul 16 11:04:30.569534 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:30.569502    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-2db6t"
Jul 16 11:04:32.143033 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:32.142983    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/19a1fad5-ffe8-43e3-8c58-028cf08b6836-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kqhbl\" (UID: \"19a1fad5-ffe8-43e3-8c58-028cf08b6836\") " pod="openshift-insights/insights-runtime-extractor-kqhbl"
Jul 16 11:04:32.145369 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:32.145347    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/19a1fad5-ffe8-43e3-8c58-028cf08b6836-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kqhbl\" (UID: \"19a1fad5-ffe8-43e3-8c58-028cf08b6836\") " pod="openshift-insights/insights-runtime-extractor-kqhbl"
Jul 16 11:04:32.269410 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:32.269373    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-lhr85\""
Jul 16 11:04:32.277444 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:32.277417    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-kqhbl"
Jul 16 11:04:32.407753 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:32.407723    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-kqhbl"]
Jul 16 11:04:32.410896 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:04:32.410868    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19a1fad5_ffe8_43e3_8c58_028cf08b6836.slice/crio-12a35d4691d8a61c093e6b63db6d48f304d2b9bf5dc7a818a3b642464045113c WatchSource:0}: Error finding container 12a35d4691d8a61c093e6b63db6d48f304d2b9bf5dc7a818a3b642464045113c: Status 404 returned error can't find the container with id 12a35d4691d8a61c093e6b63db6d48f304d2b9bf5dc7a818a3b642464045113c
Jul 16 11:04:32.732660 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:32.732568    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-kqhbl" event={"ID":"19a1fad5-ffe8-43e3-8c58-028cf08b6836","Type":"ContainerStarted","Data":"12a35d4691d8a61c093e6b63db6d48f304d2b9bf5dc7a818a3b642464045113c"}
Jul 16 11:04:34.872933 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:34.872884    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d4e7c597-9836-4a29-8591-e56938995710-metrics-certs\") pod \"network-metrics-daemon-qcl5d\" (UID: \"d4e7c597-9836-4a29-8591-e56938995710\") " pod="openshift-multus/network-metrics-daemon-qcl5d"
Jul 16 11:04:34.875520 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:34.875497    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 16 11:04:34.885924 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:34.885894    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d4e7c597-9836-4a29-8591-e56938995710-metrics-certs\") pod \"network-metrics-daemon-qcl5d\" (UID: \"d4e7c597-9836-4a29-8591-e56938995710\") " pod="openshift-multus/network-metrics-daemon-qcl5d"
Jul 16 11:04:34.974465 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:34.974430    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c88b6ce0-4926-4de9-942e-7f1b30c33e9e-original-pull-secret\") pod \"global-pull-secret-syncer-hlm2s\" (UID: \"c88b6ce0-4926-4de9-942e-7f1b30c33e9e\") " pod="kube-system/global-pull-secret-syncer-hlm2s"
Jul 16 11:04:34.977565 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:34.977543    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 16 11:04:34.988022 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:34.987997    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c88b6ce0-4926-4de9-942e-7f1b30c33e9e-original-pull-secret\") pod \"global-pull-secret-syncer-hlm2s\" (UID: \"c88b6ce0-4926-4de9-942e-7f1b30c33e9e\") " pod="kube-system/global-pull-secret-syncer-hlm2s"
Jul 16 11:04:35.074881 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:35.074844    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2kcc2\" (UniqueName: \"kubernetes.io/projected/3d0ee8e8-4018-4963-b553-af9b6ecb0f80-kube-api-access-2kcc2\") pod \"network-check-target-rc7w4\" (UID: \"3d0ee8e8-4018-4963-b553-af9b6ecb0f80\") " pod="openshift-network-diagnostics/network-check-target-rc7w4"
Jul 16 11:04:35.078059 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:35.078027    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2kcc2\" (UniqueName: \"kubernetes.io/projected/3d0ee8e8-4018-4963-b553-af9b6ecb0f80-kube-api-access-2kcc2\") pod \"network-check-target-rc7w4\" (UID: \"3d0ee8e8-4018-4963-b553-af9b6ecb0f80\") " pod="openshift-network-diagnostics/network-check-target-rc7w4"
Jul 16 11:04:35.139148 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:35.139064    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hlm2s"
Jul 16 11:04:35.147342 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:35.147319    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-49sx5\""
Jul 16 11:04:35.154018 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:35.153993    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-rdfxb\""
Jul 16 11:04:35.154997 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:35.154971    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qcl5d"
Jul 16 11:04:35.162339 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:35.162311    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rc7w4"
Jul 16 11:04:37.416352 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:37.416151    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-qcl5d"]
Jul 16 11:04:37.637803 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:04:37.637716    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4e7c597_9836_4a29_8591_e56938995710.slice/crio-6b772704b3ce7ecb4a970dee3cf7c9897046f49f7d30d6dbaf71aa32c2a868db WatchSource:0}: Error finding container 6b772704b3ce7ecb4a970dee3cf7c9897046f49f7d30d6dbaf71aa32c2a868db: Status 404 returned error can't find the container with id 6b772704b3ce7ecb4a970dee3cf7c9897046f49f7d30d6dbaf71aa32c2a868db
Jul 16 11:04:37.700459 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:37.700424    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2ae4a146-99bf-422f-982f-0dab385b060b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-9ztqs\" (UID: \"2ae4a146-99bf-422f-982f-0dab385b060b\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-9ztqs"
Jul 16 11:04:37.700635 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:37.700495    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/921e629b-d5a3-4bf6-8bad-0131d0a812ea-metrics-tls\") pod \"dns-default-5649t\" (UID: \"921e629b-d5a3-4bf6-8bad-0131d0a812ea\") " pod="openshift-dns/dns-default-5649t"
Jul 16 11:04:37.700635 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:37.700560    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ec3eb24d-8f3c-40b7-bfe9-a34fccdc20cd-service-ca-bundle\") pod \"router-default-76cb9bc7db-x2cnj\" (UID: \"ec3eb24d-8f3c-40b7-bfe9-a34fccdc20cd\") " pod="openshift-ingress/router-default-76cb9bc7db-x2cnj"
Jul 16 11:04:37.700635 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:37.700586    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6a4b5dde-f664-4d97-889e-bfe4d4df1416-registry-tls\") pod \"image-registry-65f68df599-zw6jf\" (UID: \"6a4b5dde-f664-4d97-889e-bfe4d4df1416\") " pod="openshift-image-registry/image-registry-65f68df599-zw6jf"
Jul 16 11:04:37.700635 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:37.700630    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ec3eb24d-8f3c-40b7-bfe9-a34fccdc20cd-metrics-certs\") pod \"router-default-76cb9bc7db-x2cnj\" (UID: \"ec3eb24d-8f3c-40b7-bfe9-a34fccdc20cd\") " pod="openshift-ingress/router-default-76cb9bc7db-x2cnj"
Jul 16 11:04:37.701729 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:37.701679    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ec3eb24d-8f3c-40b7-bfe9-a34fccdc20cd-service-ca-bundle\") pod \"router-default-76cb9bc7db-x2cnj\" (UID: \"ec3eb24d-8f3c-40b7-bfe9-a34fccdc20cd\") " pod="openshift-ingress/router-default-76cb9bc7db-x2cnj"
Jul 16 11:04:37.703579 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:37.703527    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/921e629b-d5a3-4bf6-8bad-0131d0a812ea-metrics-tls\") pod \"dns-default-5649t\" (UID: \"921e629b-d5a3-4bf6-8bad-0131d0a812ea\") " pod="openshift-dns/dns-default-5649t"
Jul 16 11:04:37.703706 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:37.703578    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ec3eb24d-8f3c-40b7-bfe9-a34fccdc20cd-metrics-certs\") pod \"router-default-76cb9bc7db-x2cnj\" (UID: \"ec3eb24d-8f3c-40b7-bfe9-a34fccdc20cd\") " pod="openshift-ingress/router-default-76cb9bc7db-x2cnj"
Jul 16 11:04:37.704078 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:37.704057    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6a4b5dde-f664-4d97-889e-bfe4d4df1416-registry-tls\") pod \"image-registry-65f68df599-zw6jf\" (UID: \"6a4b5dde-f664-4d97-889e-bfe4d4df1416\") " pod="openshift-image-registry/image-registry-65f68df599-zw6jf"
Jul 16 11:04:37.704491 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:37.704467    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2ae4a146-99bf-422f-982f-0dab385b060b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-9ztqs\" (UID: \"2ae4a146-99bf-422f-982f-0dab385b060b\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-9ztqs"
Jul 16 11:04:37.753289 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:37.752803    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-qcl5d" event={"ID":"d4e7c597-9836-4a29-8591-e56938995710","Type":"ContainerStarted","Data":"6b772704b3ce7ecb4a970dee3cf7c9897046f49f7d30d6dbaf71aa32c2a868db"}
Jul 16 11:04:37.778852 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:37.778817    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-rc7w4"]
Jul 16 11:04:37.783188 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:04:37.783152    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d0ee8e8_4018_4963_b553_af9b6ecb0f80.slice/crio-6a42eabd2a211e02d95f81a2806153adc31040dead5e0bfa206c729b30132acb WatchSource:0}: Error finding container 6a42eabd2a211e02d95f81a2806153adc31040dead5e0bfa206c729b30132acb: Status 404 returned error can't find the container with id 6a42eabd2a211e02d95f81a2806153adc31040dead5e0bfa206c729b30132acb
Jul 16 11:04:37.796656 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:37.796626    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-hlm2s"]
Jul 16 11:04:37.799260 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:04:37.799233    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc88b6ce0_4926_4de9_942e_7f1b30c33e9e.slice/crio-4bd3fe329aea48b36d5f309a83b7f58fd28f5b88c8c72fefd0ebe97fbfdaebea WatchSource:0}: Error finding container 4bd3fe329aea48b36d5f309a83b7f58fd28f5b88c8c72fefd0ebe97fbfdaebea: Status 404 returned error can't find the container with id 4bd3fe329aea48b36d5f309a83b7f58fd28f5b88c8c72fefd0ebe97fbfdaebea
Jul 16 11:04:37.844117 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:37.844092    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-2nzj5\""
Jul 16 11:04:37.851932 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:37.851905    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-65f68df599-zw6jf"
Jul 16 11:04:37.855472 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:37.855448    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-jfl46\""
Jul 16 11:04:37.863525 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:37.863498    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-9ztqs"
Jul 16 11:04:37.871181 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:37.871162    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-jsj9w\""
Jul 16 11:04:37.879115 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:37.879086    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-76cb9bc7db-x2cnj"
Jul 16 11:04:37.895859 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:37.895738    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-kmdfj\""
Jul 16 11:04:37.904098 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:37.904074    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-5649t"
Jul 16 11:04:38.074630 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:38.074581    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-65f68df599-zw6jf"]
Jul 16 11:04:38.077995 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:04:38.077955    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a4b5dde_f664_4d97_889e_bfe4d4df1416.slice/crio-40445f85624a9a6668acba8c6f028ab2465fc00cbb1dbffb0c9f2afb65d2790c WatchSource:0}: Error finding container 40445f85624a9a6668acba8c6f028ab2465fc00cbb1dbffb0c9f2afb65d2790c: Status 404 returned error can't find the container with id 40445f85624a9a6668acba8c6f028ab2465fc00cbb1dbffb0c9f2afb65d2790c
Jul 16 11:04:38.101333 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:38.101298    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-9ztqs"]
Jul 16 11:04:38.107513 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:04:38.106477    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ae4a146_99bf_422f_982f_0dab385b060b.slice/crio-c67ba82c2100aa6b7f3506eba055df8cda3ee5aa3d25533ab3dd61a866d93ac1 WatchSource:0}: Error finding container c67ba82c2100aa6b7f3506eba055df8cda3ee5aa3d25533ab3dd61a866d93ac1: Status 404 returned error can't find the container with id c67ba82c2100aa6b7f3506eba055df8cda3ee5aa3d25533ab3dd61a866d93ac1
Jul 16 11:04:38.144852 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:38.142975    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-5649t"]
Jul 16 11:04:38.165168 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:38.165124    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-76cb9bc7db-x2cnj"]
Jul 16 11:04:38.167934 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:04:38.167906    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec3eb24d_8f3c_40b7_bfe9_a34fccdc20cd.slice/crio-d87ac984d344b26fd2f9d8ff3863f32bebaaa080987bc9dcc8628f97f36612cc WatchSource:0}: Error finding container d87ac984d344b26fd2f9d8ff3863f32bebaaa080987bc9dcc8628f97f36612cc: Status 404 returned error can't find the container with id d87ac984d344b26fd2f9d8ff3863f32bebaaa080987bc9dcc8628f97f36612cc
Jul 16 11:04:38.678913 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:38.677620    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-68c9cbb497-755tx"]
Jul 16 11:04:38.689379 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:38.688892    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-68c9cbb497-755tx"
Jul 16 11:04:38.695724 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:38.694132    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-dm6kf\""
Jul 16 11:04:38.695724 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:38.694364    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 16 11:04:38.695724 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:38.695112    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 16 11:04:38.701364 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:38.701332    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-68c9cbb497-755tx"]
Jul 16 11:04:38.761790 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:38.761697    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-kqhbl" event={"ID":"19a1fad5-ffe8-43e3-8c58-028cf08b6836","Type":"ContainerStarted","Data":"d60fd18786dd6c19208142c5a21002045b7ae3dd5b71dea11fbcb7739172209a"}
Jul 16 11:04:38.764482 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:38.763832    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-65f68df599-zw6jf" event={"ID":"6a4b5dde-f664-4d97-889e-bfe4d4df1416","Type":"ContainerStarted","Data":"c8b43408320a38689e588fb042ef072b432fc519b8fa8934f8fcb796cd12d7d6"}
Jul 16 11:04:38.764482 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:38.763880    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-65f68df599-zw6jf" event={"ID":"6a4b5dde-f664-4d97-889e-bfe4d4df1416","Type":"ContainerStarted","Data":"40445f85624a9a6668acba8c6f028ab2465fc00cbb1dbffb0c9f2afb65d2790c"}
Jul 16 11:04:38.764482 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:38.764044    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-65f68df599-zw6jf"
Jul 16 11:04:38.765662 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:38.765568    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-hlm2s" event={"ID":"c88b6ce0-4926-4de9-942e-7f1b30c33e9e","Type":"ContainerStarted","Data":"4bd3fe329aea48b36d5f309a83b7f58fd28f5b88c8c72fefd0ebe97fbfdaebea"}
Jul 16 11:04:38.770871 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:38.770260    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-76cb9bc7db-x2cnj" event={"ID":"ec3eb24d-8f3c-40b7-bfe9-a34fccdc20cd","Type":"ContainerStarted","Data":"50e3c2bc65dd826cb5af7566ce214fc2cf32a31716fc2bd475c5982b5a9c1d9c"}
Jul 16 11:04:38.770871 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:38.770303    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-76cb9bc7db-x2cnj" event={"ID":"ec3eb24d-8f3c-40b7-bfe9-a34fccdc20cd","Type":"ContainerStarted","Data":"d87ac984d344b26fd2f9d8ff3863f32bebaaa080987bc9dcc8628f97f36612cc"}
Jul 16 11:04:38.775828 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:38.775753    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-5649t" event={"ID":"921e629b-d5a3-4bf6-8bad-0131d0a812ea","Type":"ContainerStarted","Data":"74d4cd3055f1b8acb38136f02ad19e91ea779005b7c86f5ec6599f4e532e7949"}
Jul 16 11:04:38.778808 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:38.778710    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-9ztqs" event={"ID":"2ae4a146-99bf-422f-982f-0dab385b060b","Type":"ContainerStarted","Data":"c67ba82c2100aa6b7f3506eba055df8cda3ee5aa3d25533ab3dd61a866d93ac1"}
Jul 16 11:04:38.790350 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:38.789665    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-rc7w4" event={"ID":"3d0ee8e8-4018-4963-b553-af9b6ecb0f80","Type":"ContainerStarted","Data":"0edc94d881b50bd427f12c87ae8be4752166f6a5c2cdaf4c85c0bf14ec96931b"}
Jul 16 11:04:38.790350 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:38.789721    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-rc7w4" event={"ID":"3d0ee8e8-4018-4963-b553-af9b6ecb0f80","Type":"ContainerStarted","Data":"6a42eabd2a211e02d95f81a2806153adc31040dead5e0bfa206c729b30132acb"}
Jul 16 11:04:38.790350 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:38.790302    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-rc7w4"
Jul 16 11:04:38.813317 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:38.812047    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-65f68df599-zw6jf" podStartSLOduration=68.812026835 podStartE2EDuration="1m8.812026835s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 11:03:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 11:04:38.811235171 +0000 UTC m=+71.445456626" watchObservedRunningTime="2026-07-16 11:04:38.812026835 +0000 UTC m=+71.446248290"
Jul 16 11:04:38.814003 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:38.813870    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vpsmn\" (UniqueName: \"kubernetes.io/projected/3c28c239-327f-4f45-9bf8-c6cee143cdcb-kube-api-access-vpsmn\") pod \"downloads-68c9cbb497-755tx\" (UID: \"3c28c239-327f-4f45-9bf8-c6cee143cdcb\") " pod="openshift-console/downloads-68c9cbb497-755tx"
Jul 16 11:04:38.872006 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:38.870407    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-76cb9bc7db-x2cnj" podStartSLOduration=58.870386109 podStartE2EDuration="58.870386109s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 11:03:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 11:04:38.869633512 +0000 UTC m=+71.503855014" watchObservedRunningTime="2026-07-16 11:04:38.870386109 +0000 UTC m=+71.504607564"
Jul 16 11:04:38.872006 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:38.870890    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-rc7w4" podStartSLOduration=68.87087807 podStartE2EDuration="1m8.87087807s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 11:03:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 11:04:38.840920669 +0000 UTC m=+71.475142126" watchObservedRunningTime="2026-07-16 11:04:38.87087807 +0000 UTC m=+71.505099523"
Jul 16 11:04:38.879682 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:38.879626    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-76cb9bc7db-x2cnj"
Jul 16 11:04:38.884139 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:38.884058    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-76cb9bc7db-x2cnj"
Jul 16 11:04:38.918669 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:38.915585    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vpsmn\" (UniqueName: \"kubernetes.io/projected/3c28c239-327f-4f45-9bf8-c6cee143cdcb-kube-api-access-vpsmn\") pod \"downloads-68c9cbb497-755tx\" (UID: \"3c28c239-327f-4f45-9bf8-c6cee143cdcb\") " pod="openshift-console/downloads-68c9cbb497-755tx"
Jul 16 11:04:38.935728 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:38.935623    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vpsmn\" (UniqueName: \"kubernetes.io/projected/3c28c239-327f-4f45-9bf8-c6cee143cdcb-kube-api-access-vpsmn\") pod \"downloads-68c9cbb497-755tx\" (UID: \"3c28c239-327f-4f45-9bf8-c6cee143cdcb\") " pod="openshift-console/downloads-68c9cbb497-755tx"
Jul 16 11:04:39.008433 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:39.008139    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-68c9cbb497-755tx"
Jul 16 11:04:39.214730 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:39.213456    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-68c9cbb497-755tx"]
Jul 16 11:04:39.221488 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:04:39.221439    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c28c239_327f_4f45_9bf8_c6cee143cdcb.slice/crio-58598946db027b8257f777f23c0113246530ca5e9f2996d9c14879937da5bfa2 WatchSource:0}: Error finding container 58598946db027b8257f777f23c0113246530ca5e9f2996d9c14879937da5bfa2: Status 404 returned error can't find the container with id 58598946db027b8257f777f23c0113246530ca5e9f2996d9c14879937da5bfa2
Jul 16 11:04:39.543367 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:39.543281    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-78b57b55c6-blcck"]
Jul 16 11:04:39.575391 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:39.575353    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-78b57b55c6-blcck"]
Jul 16 11:04:39.575558 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:39.575435    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-78b57b55c6-blcck"
Jul 16 11:04:39.580469 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:39.579883    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 16 11:04:39.580469 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:39.580215    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 16 11:04:39.584220 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:39.581035    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 16 11:04:39.584220 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:39.581251    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 16 11:04:39.584220 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:39.581438    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 16 11:04:39.584220 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:39.581684    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-bst9c\""
Jul 16 11:04:39.724631 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:39.724398    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cdd147a4-74f1-4ba7-a9dc-527960187ddc-console-serving-cert\") pod \"console-78b57b55c6-blcck\" (UID: \"cdd147a4-74f1-4ba7-a9dc-527960187ddc\") " pod="openshift-console/console-78b57b55c6-blcck"
Jul 16 11:04:39.724631 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:39.724450    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cdd147a4-74f1-4ba7-a9dc-527960187ddc-service-ca\") pod \"console-78b57b55c6-blcck\" (UID: \"cdd147a4-74f1-4ba7-a9dc-527960187ddc\") " pod="openshift-console/console-78b57b55c6-blcck"
Jul 16 11:04:39.724631 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:39.724480    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cdd147a4-74f1-4ba7-a9dc-527960187ddc-console-oauth-config\") pod \"console-78b57b55c6-blcck\" (UID: \"cdd147a4-74f1-4ba7-a9dc-527960187ddc\") " pod="openshift-console/console-78b57b55c6-blcck"
Jul 16 11:04:39.725329 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:39.724682    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cdd147a4-74f1-4ba7-a9dc-527960187ddc-oauth-serving-cert\") pod \"console-78b57b55c6-blcck\" (UID: \"cdd147a4-74f1-4ba7-a9dc-527960187ddc\") " pod="openshift-console/console-78b57b55c6-blcck"
Jul 16 11:04:39.725329 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:39.724728    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kxmbx\" (UniqueName: \"kubernetes.io/projected/cdd147a4-74f1-4ba7-a9dc-527960187ddc-kube-api-access-kxmbx\") pod \"console-78b57b55c6-blcck\" (UID: \"cdd147a4-74f1-4ba7-a9dc-527960187ddc\") " pod="openshift-console/console-78b57b55c6-blcck"
Jul 16 11:04:39.725329 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:39.724766    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cdd147a4-74f1-4ba7-a9dc-527960187ddc-console-config\") pod \"console-78b57b55c6-blcck\" (UID: \"cdd147a4-74f1-4ba7-a9dc-527960187ddc\") " pod="openshift-console/console-78b57b55c6-blcck"
Jul 16 11:04:39.798438 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:39.798339    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-68c9cbb497-755tx" event={"ID":"3c28c239-327f-4f45-9bf8-c6cee143cdcb","Type":"ContainerStarted","Data":"58598946db027b8257f777f23c0113246530ca5e9f2996d9c14879937da5bfa2"}
Jul 16 11:04:39.798438 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:39.798430    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-76cb9bc7db-x2cnj"
Jul 16 11:04:39.800219 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:39.800050    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-76cb9bc7db-x2cnj"
Jul 16 11:04:39.825937 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:39.825898    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cdd147a4-74f1-4ba7-a9dc-527960187ddc-oauth-serving-cert\") pod \"console-78b57b55c6-blcck\" (UID: \"cdd147a4-74f1-4ba7-a9dc-527960187ddc\") " pod="openshift-console/console-78b57b55c6-blcck"
Jul 16 11:04:39.826133 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:39.825952    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kxmbx\" (UniqueName: \"kubernetes.io/projected/cdd147a4-74f1-4ba7-a9dc-527960187ddc-kube-api-access-kxmbx\") pod \"console-78b57b55c6-blcck\" (UID: \"cdd147a4-74f1-4ba7-a9dc-527960187ddc\") " pod="openshift-console/console-78b57b55c6-blcck"
Jul 16 11:04:39.826133 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:39.825988    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cdd147a4-74f1-4ba7-a9dc-527960187ddc-console-config\") pod \"console-78b57b55c6-blcck\" (UID: \"cdd147a4-74f1-4ba7-a9dc-527960187ddc\") " pod="openshift-console/console-78b57b55c6-blcck"
Jul 16 11:04:39.826133 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:39.826050    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cdd147a4-74f1-4ba7-a9dc-527960187ddc-console-serving-cert\") pod \"console-78b57b55c6-blcck\" (UID: \"cdd147a4-74f1-4ba7-a9dc-527960187ddc\") " pod="openshift-console/console-78b57b55c6-blcck"
Jul 16 11:04:39.826133 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:39.826075    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cdd147a4-74f1-4ba7-a9dc-527960187ddc-service-ca\") pod \"console-78b57b55c6-blcck\" (UID: \"cdd147a4-74f1-4ba7-a9dc-527960187ddc\") " pod="openshift-console/console-78b57b55c6-blcck"
Jul 16 11:04:39.826133 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:39.826102    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cdd147a4-74f1-4ba7-a9dc-527960187ddc-console-oauth-config\") pod \"console-78b57b55c6-blcck\" (UID: \"cdd147a4-74f1-4ba7-a9dc-527960187ddc\") " pod="openshift-console/console-78b57b55c6-blcck"
Jul 16 11:04:39.830090 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:39.827257    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cdd147a4-74f1-4ba7-a9dc-527960187ddc-console-config\") pod \"console-78b57b55c6-blcck\" (UID: \"cdd147a4-74f1-4ba7-a9dc-527960187ddc\") " pod="openshift-console/console-78b57b55c6-blcck"
Jul 16 11:04:39.830090 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:39.827825    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cdd147a4-74f1-4ba7-a9dc-527960187ddc-oauth-serving-cert\") pod \"console-78b57b55c6-blcck\" (UID: \"cdd147a4-74f1-4ba7-a9dc-527960187ddc\") " pod="openshift-console/console-78b57b55c6-blcck"
Jul 16 11:04:39.830385 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:39.830175    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cdd147a4-74f1-4ba7-a9dc-527960187ddc-service-ca\") pod \"console-78b57b55c6-blcck\" (UID: \"cdd147a4-74f1-4ba7-a9dc-527960187ddc\") " pod="openshift-console/console-78b57b55c6-blcck"
Jul 16 11:04:39.833899 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:39.833856    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cdd147a4-74f1-4ba7-a9dc-527960187ddc-console-serving-cert\") pod \"console-78b57b55c6-blcck\" (UID: \"cdd147a4-74f1-4ba7-a9dc-527960187ddc\") " pod="openshift-console/console-78b57b55c6-blcck"
Jul 16 11:04:39.835152 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:39.835094    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cdd147a4-74f1-4ba7-a9dc-527960187ddc-console-oauth-config\") pod \"console-78b57b55c6-blcck\" (UID: \"cdd147a4-74f1-4ba7-a9dc-527960187ddc\") " pod="openshift-console/console-78b57b55c6-blcck"
Jul 16 11:04:39.847028 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:39.846968    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kxmbx\" (UniqueName: \"kubernetes.io/projected/cdd147a4-74f1-4ba7-a9dc-527960187ddc-kube-api-access-kxmbx\") pod \"console-78b57b55c6-blcck\" (UID: \"cdd147a4-74f1-4ba7-a9dc-527960187ddc\") " pod="openshift-console/console-78b57b55c6-blcck"
Jul 16 11:04:39.893508 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:39.893247    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-78b57b55c6-blcck"
Jul 16 11:04:40.633801 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:40.633760    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9fceac27-6fd9-4091-af74-783b21616981-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-vgn2h\" (UID: \"9fceac27-6fd9-4091-af74-783b21616981\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-vgn2h"
Jul 16 11:04:40.638093 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:40.638022    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9fceac27-6fd9-4091-af74-783b21616981-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-vgn2h\" (UID: \"9fceac27-6fd9-4091-af74-783b21616981\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-vgn2h"
Jul 16 11:04:40.787558 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:40.787524    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-fn5b5\""
Jul 16 11:04:40.795262 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:40.795229    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-vgn2h"
Jul 16 11:04:45.575448 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:45.575353    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-78b57b55c6-blcck"]
Jul 16 11:04:46.407095 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:04:46.407059    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcdd147a4_74f1_4ba7_a9dc_527960187ddc.slice/crio-599c7ddcd2cc59591e83d43527ba030ab9e211c4a30960ca0bd7f18ba3839817 WatchSource:0}: Error finding container 599c7ddcd2cc59591e83d43527ba030ab9e211c4a30960ca0bd7f18ba3839817: Status 404 returned error can't find the container with id 599c7ddcd2cc59591e83d43527ba030ab9e211c4a30960ca0bd7f18ba3839817
Jul 16 11:04:46.559058 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:46.558968    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-vgn2h"]
Jul 16 11:04:46.602448 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:04:46.602398    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9fceac27_6fd9_4091_af74_783b21616981.slice/crio-7ae4d32e365de4a985be073379d0eaac075c2ff656dd6ca4a409da7855361106 WatchSource:0}: Error finding container 7ae4d32e365de4a985be073379d0eaac075c2ff656dd6ca4a409da7855361106: Status 404 returned error can't find the container with id 7ae4d32e365de4a985be073379d0eaac075c2ff656dd6ca4a409da7855361106
Jul 16 11:04:46.838048 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:46.837952    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6cf856758c-vgn2h" event={"ID":"9fceac27-6fd9-4091-af74-783b21616981","Type":"ContainerStarted","Data":"7ae4d32e365de4a985be073379d0eaac075c2ff656dd6ca4a409da7855361106"}
Jul 16 11:04:46.840218 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:46.840143    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-5649t" event={"ID":"921e629b-d5a3-4bf6-8bad-0131d0a812ea","Type":"ContainerStarted","Data":"ff48bf503e85adc19b76748c4e577a4531d27174f137f2899d2aab64fde8f348"}
Jul 16 11:04:46.847159 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:46.847046    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-9ztqs" event={"ID":"2ae4a146-99bf-422f-982f-0dab385b060b","Type":"ContainerStarted","Data":"48b3b091bce3c076718914c661fc054a207b8be5a45f66f08af5a75af33ac6d0"}
Jul 16 11:04:46.855420 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:46.855335    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-78b57b55c6-blcck" event={"ID":"cdd147a4-74f1-4ba7-a9dc-527960187ddc","Type":"ContainerStarted","Data":"599c7ddcd2cc59591e83d43527ba030ab9e211c4a30960ca0bd7f18ba3839817"}
Jul 16 11:04:46.859743 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:46.859620    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-qcl5d" event={"ID":"d4e7c597-9836-4a29-8591-e56938995710","Type":"ContainerStarted","Data":"b8e7c85689333e1bdeb02b39430562254012dd462c8d4fefba7e1f339fff6598"}
Jul 16 11:04:46.866388 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:46.865315    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-9ztqs" podStartSLOduration=58.720259657 podStartE2EDuration="1m6.865294373s" totalImagesPullingTime="8.145034716s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 11:03:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 11:04:46.86395829 +0000 UTC m=+79.498179756" watchObservedRunningTime="2026-07-16 11:04:46.865294373 +0000 UTC m=+79.499515828"
Jul 16 11:04:47.866823 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:47.866739    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-kqhbl" event={"ID":"19a1fad5-ffe8-43e3-8c58-028cf08b6836","Type":"ContainerStarted","Data":"716d9f17b9e2b83beb5cc2cf192bff3a9a4cfb0a490526c66df9b8ce976da78c"}
Jul 16 11:04:47.869038 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:47.868907    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-hlm2s" event={"ID":"c88b6ce0-4926-4de9-942e-7f1b30c33e9e","Type":"ContainerStarted","Data":"4438b1678cc4951eaed5498e145c7e96e670465a574af8b51c6307c60ec422ab"}
Jul 16 11:04:47.871835 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:47.871267    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-5649t" event={"ID":"921e629b-d5a3-4bf6-8bad-0131d0a812ea","Type":"ContainerStarted","Data":"42fa1062a58d66347735e37c07d71e5e3093d486d2a2e14969dbb84a3516e2c8"}
Jul 16 11:04:47.871835 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:47.871331    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-5649t"
Jul 16 11:04:47.874085 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:47.874053    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-qcl5d" event={"ID":"d4e7c597-9836-4a29-8591-e56938995710","Type":"ContainerStarted","Data":"10d6012ffeecef50a7e8453cb1b6a3e139f6c52ffd224f97e936aecd46265986"}
Jul 16 11:04:47.888476 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:47.888408    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-hlm2s" podStartSLOduration=69.048425394 podStartE2EDuration="1m17.88838481s" totalImagesPullingTime="8.839959416s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 11:03:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 11:04:47.887787913 +0000 UTC m=+80.522009361" watchObservedRunningTime="2026-07-16 11:04:47.88838481 +0000 UTC m=+80.522606283"
Jul 16 11:04:47.911514 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:47.911449    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-5649t" podStartSLOduration=34.805289898 podStartE2EDuration="42.911428282s" totalImagesPullingTime="8.106138384s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 11:04:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 11:04:47.911040288 +0000 UTC m=+80.545261766" watchObservedRunningTime="2026-07-16 11:04:47.911428282 +0000 UTC m=+80.545649737"
Jul 16 11:04:47.927510 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:47.927453    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-qcl5d" podStartSLOduration=70.273081847 podStartE2EDuration="1m17.927435146s" totalImagesPullingTime="7.654353299s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 11:03:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 11:04:47.926977746 +0000 UTC m=+80.561199203" watchObservedRunningTime="2026-07-16 11:04:47.927435146 +0000 UTC m=+80.561656603"
Jul 16 11:04:55.930849 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:55.930706    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-6n2hb"]
Jul 16 11:04:56.063577 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:56.063537    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-6n2hb"
Jul 16 11:04:56.066587 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:56.066528    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 16 11:04:56.066787 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:56.066673    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 16 11:04:56.066787 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:56.066713    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-898zm\""
Jul 16 11:04:56.067587 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:56.067565    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 16 11:04:56.067881 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:56.067853    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 16 11:04:56.073831 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:56.073587    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/44df078b-d929-4693-bce4-5ebde6a53923-node-exporter-accelerators-collector-config\") pod \"node-exporter-6n2hb\" (UID: \"44df078b-d929-4693-bce4-5ebde6a53923\") " pod="openshift-monitoring/node-exporter-6n2hb"
Jul 16 11:04:56.073831 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:56.073650    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ft5t5\" (UniqueName: \"kubernetes.io/projected/44df078b-d929-4693-bce4-5ebde6a53923-kube-api-access-ft5t5\") pod \"node-exporter-6n2hb\" (UID: \"44df078b-d929-4693-bce4-5ebde6a53923\") " pod="openshift-monitoring/node-exporter-6n2hb"
Jul 16 11:04:56.073831 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:56.073684    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/44df078b-d929-4693-bce4-5ebde6a53923-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-6n2hb\" (UID: \"44df078b-d929-4693-bce4-5ebde6a53923\") " pod="openshift-monitoring/node-exporter-6n2hb"
Jul 16 11:04:56.073831 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:56.073714    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/44df078b-d929-4693-bce4-5ebde6a53923-node-exporter-wtmp\") pod \"node-exporter-6n2hb\" (UID: \"44df078b-d929-4693-bce4-5ebde6a53923\") " pod="openshift-monitoring/node-exporter-6n2hb"
Jul 16 11:04:56.073831 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:56.073772    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/44df078b-d929-4693-bce4-5ebde6a53923-metrics-client-ca\") pod \"node-exporter-6n2hb\" (UID: \"44df078b-d929-4693-bce4-5ebde6a53923\") " pod="openshift-monitoring/node-exporter-6n2hb"
Jul 16 11:04:56.073831 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:56.073805    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/44df078b-d929-4693-bce4-5ebde6a53923-root\") pod \"node-exporter-6n2hb\" (UID: \"44df078b-d929-4693-bce4-5ebde6a53923\") " pod="openshift-monitoring/node-exporter-6n2hb"
Jul 16 11:04:56.073831 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:56.073830    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/44df078b-d929-4693-bce4-5ebde6a53923-sys\") pod \"node-exporter-6n2hb\" (UID: \"44df078b-d929-4693-bce4-5ebde6a53923\") " pod="openshift-monitoring/node-exporter-6n2hb"
Jul 16 11:04:56.074171 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:56.073864    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/44df078b-d929-4693-bce4-5ebde6a53923-node-exporter-tls\") pod \"node-exporter-6n2hb\" (UID: \"44df078b-d929-4693-bce4-5ebde6a53923\") " pod="openshift-monitoring/node-exporter-6n2hb"
Jul 16 11:04:56.074171 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:56.073888    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/44df078b-d929-4693-bce4-5ebde6a53923-node-exporter-textfile\") pod \"node-exporter-6n2hb\" (UID: \"44df078b-d929-4693-bce4-5ebde6a53923\") " pod="openshift-monitoring/node-exporter-6n2hb"
Jul 16 11:04:56.175099 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:56.175047    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/44df078b-d929-4693-bce4-5ebde6a53923-node-exporter-accelerators-collector-config\") pod \"node-exporter-6n2hb\" (UID: \"44df078b-d929-4693-bce4-5ebde6a53923\") " pod="openshift-monitoring/node-exporter-6n2hb"
Jul 16 11:04:56.175289 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:56.175110    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ft5t5\" (UniqueName: \"kubernetes.io/projected/44df078b-d929-4693-bce4-5ebde6a53923-kube-api-access-ft5t5\") pod \"node-exporter-6n2hb\" (UID: \"44df078b-d929-4693-bce4-5ebde6a53923\") " pod="openshift-monitoring/node-exporter-6n2hb"
Jul 16 11:04:56.175289 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:56.175143    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/44df078b-d929-4693-bce4-5ebde6a53923-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-6n2hb\" (UID: \"44df078b-d929-4693-bce4-5ebde6a53923\") " pod="openshift-monitoring/node-exporter-6n2hb"
Jul 16 11:04:56.175402 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:56.175293    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/44df078b-d929-4693-bce4-5ebde6a53923-node-exporter-wtmp\") pod \"node-exporter-6n2hb\" (UID: \"44df078b-d929-4693-bce4-5ebde6a53923\") " pod="openshift-monitoring/node-exporter-6n2hb"
Jul 16 11:04:56.175402 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:56.175341    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/44df078b-d929-4693-bce4-5ebde6a53923-metrics-client-ca\") pod \"node-exporter-6n2hb\" (UID: \"44df078b-d929-4693-bce4-5ebde6a53923\") " pod="openshift-monitoring/node-exporter-6n2hb"
Jul 16 11:04:56.175402 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:56.175370    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/44df078b-d929-4693-bce4-5ebde6a53923-root\") pod \"node-exporter-6n2hb\" (UID: \"44df078b-d929-4693-bce4-5ebde6a53923\") " pod="openshift-monitoring/node-exporter-6n2hb"
Jul 16 11:04:56.175402 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:56.175395    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/44df078b-d929-4693-bce4-5ebde6a53923-sys\") pod \"node-exporter-6n2hb\" (UID: \"44df078b-d929-4693-bce4-5ebde6a53923\") " pod="openshift-monitoring/node-exporter-6n2hb"
Jul 16 11:04:56.175586 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:56.175433    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/44df078b-d929-4693-bce4-5ebde6a53923-node-exporter-tls\") pod \"node-exporter-6n2hb\" (UID: \"44df078b-d929-4693-bce4-5ebde6a53923\") " pod="openshift-monitoring/node-exporter-6n2hb"
Jul 16 11:04:56.175586 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:56.175457    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/44df078b-d929-4693-bce4-5ebde6a53923-node-exporter-textfile\") pod \"node-exporter-6n2hb\" (UID: \"44df078b-d929-4693-bce4-5ebde6a53923\") " pod="openshift-monitoring/node-exporter-6n2hb"
Jul 16 11:04:56.175586 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:56.175529    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/44df078b-d929-4693-bce4-5ebde6a53923-root\") pod \"node-exporter-6n2hb\" (UID: \"44df078b-d929-4693-bce4-5ebde6a53923\") " pod="openshift-monitoring/node-exporter-6n2hb"
Jul 16 11:04:56.175801 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:56.175654    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/44df078b-d929-4693-bce4-5ebde6a53923-node-exporter-wtmp\") pod \"node-exporter-6n2hb\" (UID: \"44df078b-d929-4693-bce4-5ebde6a53923\") " pod="openshift-monitoring/node-exporter-6n2hb"
Jul 16 11:04:56.175801 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:56.175712    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/44df078b-d929-4693-bce4-5ebde6a53923-sys\") pod \"node-exporter-6n2hb\" (UID: \"44df078b-d929-4693-bce4-5ebde6a53923\") " pod="openshift-monitoring/node-exporter-6n2hb"
Jul 16 11:04:56.175801 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:56.175771    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/44df078b-d929-4693-bce4-5ebde6a53923-node-exporter-accelerators-collector-config\") pod \"node-exporter-6n2hb\" (UID: \"44df078b-d929-4693-bce4-5ebde6a53923\") " pod="openshift-monitoring/node-exporter-6n2hb"
Jul 16 11:04:56.175956 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:56.175804    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/44df078b-d929-4693-bce4-5ebde6a53923-node-exporter-textfile\") pod \"node-exporter-6n2hb\" (UID: \"44df078b-d929-4693-bce4-5ebde6a53923\") " pod="openshift-monitoring/node-exporter-6n2hb"
Jul 16 11:04:56.176135 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:56.176093    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/44df078b-d929-4693-bce4-5ebde6a53923-metrics-client-ca\") pod \"node-exporter-6n2hb\" (UID: \"44df078b-d929-4693-bce4-5ebde6a53923\") " pod="openshift-monitoring/node-exporter-6n2hb"
Jul 16 11:04:56.178177 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:56.178155    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/44df078b-d929-4693-bce4-5ebde6a53923-node-exporter-tls\") pod \"node-exporter-6n2hb\" (UID: \"44df078b-d929-4693-bce4-5ebde6a53923\") " pod="openshift-monitoring/node-exporter-6n2hb"
Jul 16 11:04:56.178383 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:56.178348    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/44df078b-d929-4693-bce4-5ebde6a53923-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-6n2hb\" (UID: \"44df078b-d929-4693-bce4-5ebde6a53923\") " pod="openshift-monitoring/node-exporter-6n2hb"
Jul 16 11:04:56.184897 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:56.184833    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ft5t5\" (UniqueName: \"kubernetes.io/projected/44df078b-d929-4693-bce4-5ebde6a53923-kube-api-access-ft5t5\") pod \"node-exporter-6n2hb\" (UID: \"44df078b-d929-4693-bce4-5ebde6a53923\") " pod="openshift-monitoring/node-exporter-6n2hb"
Jul 16 11:04:56.376334 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:56.376292    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-6n2hb"
Jul 16 11:04:57.857482 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:57.857439    2583 patch_prober.go:28] interesting pod/image-registry-65f68df599-zw6jf container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 16 11:04:57.857945 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:57.857502    2583 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-65f68df599-zw6jf" podUID="6a4b5dde-f664-4d97-889e-bfe4d4df1416" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 16 11:04:57.883649 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:57.883617    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-5649t"
Jul 16 11:04:59.741080 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:04:59.741045    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44df078b_d929_4693_bce4_5ebde6a53923.slice/crio-cfcb2ec31c99fd91f9679d145308c2e29d28db2509eda9eedf2023066cbbcc35 WatchSource:0}: Error finding container cfcb2ec31c99fd91f9679d145308c2e29d28db2509eda9eedf2023066cbbcc35: Status 404 returned error can't find the container with id cfcb2ec31c99fd91f9679d145308c2e29d28db2509eda9eedf2023066cbbcc35
Jul 16 11:04:59.803292 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:59.803270    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-65f68df599-zw6jf"
Jul 16 11:04:59.934032 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:59.933985    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6cf856758c-vgn2h" event={"ID":"9fceac27-6fd9-4091-af74-783b21616981","Type":"ContainerStarted","Data":"2ba4d3ec1f45c7f4e9f47c38ee286808f2ba07177049db8863ce3670bbe05b54"}
Jul 16 11:04:59.935682 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:59.935651    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-78b57b55c6-blcck" event={"ID":"cdd147a4-74f1-4ba7-a9dc-527960187ddc","Type":"ContainerStarted","Data":"f91fe19cb741d71812d03323b4d3a8becb368d45721a269ff996013f305cc9ba"}
Jul 16 11:04:59.936952 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:59.936928    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6n2hb" event={"ID":"44df078b-d929-4693-bce4-5ebde6a53923","Type":"ContainerStarted","Data":"cfcb2ec31c99fd91f9679d145308c2e29d28db2509eda9eedf2023066cbbcc35"}
Jul 16 11:04:59.939165 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:59.939138    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-kqhbl" event={"ID":"19a1fad5-ffe8-43e3-8c58-028cf08b6836","Type":"ContainerStarted","Data":"13d99a92bf1ee781d464082e071815f4e9410bab1ff294392911a49c26c29b03"}
Jul 16 11:04:59.940754 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:59.940733    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-68c9cbb497-755tx" event={"ID":"3c28c239-327f-4f45-9bf8-c6cee143cdcb","Type":"ContainerStarted","Data":"52764275c7006a1cdf398fc13f1034fb1daaf4bad560fbdd5fddab5c96d53205"}
Jul 16 11:04:59.940943 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:59.940925    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-68c9cbb497-755tx"
Jul 16 11:04:59.942620 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:59.942582    2583 patch_prober.go:28] interesting pod/downloads-68c9cbb497-755tx container/download-server namespace/openshift-console: Readiness probe status=failure output="Get \"http://10.132.0.22:8080/\": dial tcp 10.132.0.22:8080: connect: connection refused" start-of-body=
Jul 16 11:04:59.942713 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:59.942641    2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console/downloads-68c9cbb497-755tx" podUID="3c28c239-327f-4f45-9bf8-c6cee143cdcb" containerName="download-server" probeResult="failure" output="Get \"http://10.132.0.22:8080/\": dial tcp 10.132.0.22:8080: connect: connection refused"
Jul 16 11:04:59.963188 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:59.963134    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6cf856758c-vgn2h" podStartSLOduration=38.848606557 podStartE2EDuration="51.963116322s" totalImagesPullingTime="13.114509765s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 11:04:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 11:04:59.961684213 +0000 UTC m=+92.595905693" watchObservedRunningTime="2026-07-16 11:04:59.963116322 +0000 UTC m=+92.597337800"
Jul 16 11:04:59.998205 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:04:59.998149    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-68c9cbb497-755tx" podStartSLOduration=1.436204046 podStartE2EDuration="21.99813173s" totalImagesPullingTime="20.561927684s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 11:04:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 11:04:59.996403968 +0000 UTC m=+92.630625461" watchObservedRunningTime="2026-07-16 11:04:59.99813173 +0000 UTC m=+92.632353182"
Jul 16 11:05:00.024518 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:05:00.024464    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-kqhbl" podStartSLOduration=16.880795675 podStartE2EDuration="44.024446215s" totalImagesPullingTime="27.14365054s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 11:04:16 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 11:05:00.023583265 +0000 UTC m=+92.657804737" watchObservedRunningTime="2026-07-16 11:05:00.024446215 +0000 UTC m=+92.658667724"
Jul 16 11:05:00.058256 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:05:00.058163    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-78b57b55c6-blcck" podStartSLOduration=7.73007193 podStartE2EDuration="21.05814179s" totalImagesPullingTime="13.32806986s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 11:04:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 11:05:00.05715429 +0000 UTC m=+92.691375746" watchObservedRunningTime="2026-07-16 11:05:00.05814179 +0000 UTC m=+92.692363250"
Jul 16 11:05:00.610168 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:05:00.610120    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-78b57b55c6-blcck"]
Jul 16 11:05:00.960660 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:05:00.960621    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-68c9cbb497-755tx"
Jul 16 11:05:01.953353 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:05:01.953312    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6n2hb" event={"ID":"44df078b-d929-4693-bce4-5ebde6a53923","Type":"ContainerStarted","Data":"c4057345707289f531fae1de1dc295f0df42babfe89394cbf55aa4f24a71f4fd"}
Jul 16 11:05:02.959234 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:05:02.959196    2583 generic.go:356] "Generic (PLEG): container finished" podID="44df078b-d929-4693-bce4-5ebde6a53923" containerID="c4057345707289f531fae1de1dc295f0df42babfe89394cbf55aa4f24a71f4fd" exitCode=0
Jul 16 11:05:02.959699 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:05:02.959292    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6n2hb" event={"ID":"44df078b-d929-4693-bce4-5ebde6a53923","Type":"ContainerDied","Data":"c4057345707289f531fae1de1dc295f0df42babfe89394cbf55aa4f24a71f4fd"}
Jul 16 11:05:03.537478 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:05:03.537385    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-65f68df599-zw6jf"]
Jul 16 11:05:03.967413 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:05:03.967376    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6n2hb" event={"ID":"44df078b-d929-4693-bce4-5ebde6a53923","Type":"ContainerStarted","Data":"523d95f65de2ca4f87f1109cc3f9d282f2dac332e2e981da6fa3d919320e1fa7"}
Jul 16 11:05:03.967891 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:05:03.967436    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6n2hb" event={"ID":"44df078b-d929-4693-bce4-5ebde6a53923","Type":"ContainerStarted","Data":"ad3297708fd5e35259fe263fea2b5c0b53d25210f1873b5b152361c3149586e8"}
Jul 16 11:05:03.988184 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:05:03.988123    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-6n2hb" podStartSLOduration=6.925182367 podStartE2EDuration="8.98810668s" totalImagesPullingTime="1.949904054s" totalInitContainerRuntime="113.020259ms" isStatefulPod=true podCreationTimestamp="2026-07-16 11:04:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 11:05:03.986416039 +0000 UTC m=+96.620637515" watchObservedRunningTime="2026-07-16 11:05:03.98810668 +0000 UTC m=+96.622328134"
Jul 16 11:05:09.894261 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:05:09.894211    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-78b57b55c6-blcck"
Jul 16 11:05:10.805326 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:05:10.805289    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-rc7w4"
Jul 16 11:05:21.039175 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:05:21.039081    2583 generic.go:356] "Generic (PLEG): container finished" podID="c839c027-e922-48f5-9940-d293b38aef05" containerID="2ab6722c18c456d276b495805cba05191a6de7f6554af801831f6d9b34620ede" exitCode=0
Jul 16 11:05:21.039175 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:05:21.039155    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-lx49b" event={"ID":"c839c027-e922-48f5-9940-d293b38aef05","Type":"ContainerDied","Data":"2ab6722c18c456d276b495805cba05191a6de7f6554af801831f6d9b34620ede"}
Jul 16 11:05:21.039777 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:05:21.039628    2583 scope.go:117] "RemoveContainer" containerID="2ab6722c18c456d276b495805cba05191a6de7f6554af801831f6d9b34620ede"
Jul 16 11:05:22.046233 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:05:22.046200    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-lx49b" event={"ID":"c839c027-e922-48f5-9940-d293b38aef05","Type":"ContainerStarted","Data":"7aeefa173cef08bb87a7d1983616b010e6432963b719c0e60ef57c59509b63f1"}
Jul 16 11:05:26.979283 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:05:26.979240    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-78b57b55c6-blcck" podUID="cdd147a4-74f1-4ba7-a9dc-527960187ddc" containerName="console" containerID="cri-o://f91fe19cb741d71812d03323b4d3a8becb368d45721a269ff996013f305cc9ba" gracePeriod=15
Jul 16 11:05:27.255222 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:05:27.255191    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-78b57b55c6-blcck"
Jul 16 11:05:27.356179 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:05:27.356137    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cdd147a4-74f1-4ba7-a9dc-527960187ddc-console-serving-cert\") pod \"cdd147a4-74f1-4ba7-a9dc-527960187ddc\" (UID: \"cdd147a4-74f1-4ba7-a9dc-527960187ddc\") "
Jul 16 11:05:27.356366 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:05:27.356199    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kxmbx\" (UniqueName: \"kubernetes.io/projected/cdd147a4-74f1-4ba7-a9dc-527960187ddc-kube-api-access-kxmbx\") pod \"cdd147a4-74f1-4ba7-a9dc-527960187ddc\" (UID: \"cdd147a4-74f1-4ba7-a9dc-527960187ddc\") "
Jul 16 11:05:27.356433 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:05:27.356387    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cdd147a4-74f1-4ba7-a9dc-527960187ddc-console-config\") pod \"cdd147a4-74f1-4ba7-a9dc-527960187ddc\" (UID: \"cdd147a4-74f1-4ba7-a9dc-527960187ddc\") "
Jul 16 11:05:27.356494 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:05:27.356446    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cdd147a4-74f1-4ba7-a9dc-527960187ddc-oauth-serving-cert\") pod \"cdd147a4-74f1-4ba7-a9dc-527960187ddc\" (UID: \"cdd147a4-74f1-4ba7-a9dc-527960187ddc\") "
Jul 16 11:05:27.356494 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:05:27.356479    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cdd147a4-74f1-4ba7-a9dc-527960187ddc-console-oauth-config\") pod \"cdd147a4-74f1-4ba7-a9dc-527960187ddc\" (UID: \"cdd147a4-74f1-4ba7-a9dc-527960187ddc\") "
Jul 16 11:05:27.356586 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:05:27.356514    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cdd147a4-74f1-4ba7-a9dc-527960187ddc-service-ca\") pod \"cdd147a4-74f1-4ba7-a9dc-527960187ddc\" (UID: \"cdd147a4-74f1-4ba7-a9dc-527960187ddc\") "
Jul 16 11:05:27.356887 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:05:27.356853    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cdd147a4-74f1-4ba7-a9dc-527960187ddc-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "cdd147a4-74f1-4ba7-a9dc-527960187ddc" (UID: "cdd147a4-74f1-4ba7-a9dc-527960187ddc"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 11:05:27.357073 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:05:27.356856    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cdd147a4-74f1-4ba7-a9dc-527960187ddc-console-config" (OuterVolumeSpecName: "console-config") pod "cdd147a4-74f1-4ba7-a9dc-527960187ddc" (UID: "cdd147a4-74f1-4ba7-a9dc-527960187ddc"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 11:05:27.357073 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:05:27.356908    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cdd147a4-74f1-4ba7-a9dc-527960187ddc-service-ca" (OuterVolumeSpecName: "service-ca") pod "cdd147a4-74f1-4ba7-a9dc-527960187ddc" (UID: "cdd147a4-74f1-4ba7-a9dc-527960187ddc"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 11:05:27.358675 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:05:27.358642    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cdd147a4-74f1-4ba7-a9dc-527960187ddc-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "cdd147a4-74f1-4ba7-a9dc-527960187ddc" (UID: "cdd147a4-74f1-4ba7-a9dc-527960187ddc"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 11:05:27.358745 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:05:27.358707    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cdd147a4-74f1-4ba7-a9dc-527960187ddc-kube-api-access-kxmbx" (OuterVolumeSpecName: "kube-api-access-kxmbx") pod "cdd147a4-74f1-4ba7-a9dc-527960187ddc" (UID: "cdd147a4-74f1-4ba7-a9dc-527960187ddc"). InnerVolumeSpecName "kube-api-access-kxmbx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 11:05:27.358745 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:05:27.358725    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cdd147a4-74f1-4ba7-a9dc-527960187ddc-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "cdd147a4-74f1-4ba7-a9dc-527960187ddc" (UID: "cdd147a4-74f1-4ba7-a9dc-527960187ddc"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 11:05:27.457129 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:05:27.457089    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kxmbx\" (UniqueName: \"kubernetes.io/projected/cdd147a4-74f1-4ba7-a9dc-527960187ddc-kube-api-access-kxmbx\") on node \"ip-10-0-129-28.ec2.internal\" DevicePath \"\""
Jul 16 11:05:27.457129 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:05:27.457123    2583 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cdd147a4-74f1-4ba7-a9dc-527960187ddc-console-config\") on node \"ip-10-0-129-28.ec2.internal\" DevicePath \"\""
Jul 16 11:05:27.457129 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:05:27.457134    2583 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cdd147a4-74f1-4ba7-a9dc-527960187ddc-oauth-serving-cert\") on node \"ip-10-0-129-28.ec2.internal\" DevicePath \"\""
Jul 16 11:05:27.457365 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:05:27.457144    2583 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cdd147a4-74f1-4ba7-a9dc-527960187ddc-console-oauth-config\") on node \"ip-10-0-129-28.ec2.internal\" DevicePath \"\""
Jul 16 11:05:27.457365 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:05:27.457153    2583 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cdd147a4-74f1-4ba7-a9dc-527960187ddc-service-ca\") on node \"ip-10-0-129-28.ec2.internal\" DevicePath \"\""
Jul 16 11:05:27.457365 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:05:27.457161    2583 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cdd147a4-74f1-4ba7-a9dc-527960187ddc-console-serving-cert\") on node \"ip-10-0-129-28.ec2.internal\" DevicePath \"\""
Jul 16 11:05:28.069963 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:05:28.069923    2583 generic.go:356] "Generic (PLEG): container finished" podID="cdd147a4-74f1-4ba7-a9dc-527960187ddc" containerID="f91fe19cb741d71812d03323b4d3a8becb368d45721a269ff996013f305cc9ba" exitCode=0
Jul 16 11:05:28.070525 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:05:28.069977    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-78b57b55c6-blcck"
Jul 16 11:05:28.070525 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:05:28.070020    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-78b57b55c6-blcck" event={"ID":"cdd147a4-74f1-4ba7-a9dc-527960187ddc","Type":"ContainerDied","Data":"f91fe19cb741d71812d03323b4d3a8becb368d45721a269ff996013f305cc9ba"}
Jul 16 11:05:28.070525 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:05:28.070087    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-78b57b55c6-blcck" event={"ID":"cdd147a4-74f1-4ba7-a9dc-527960187ddc","Type":"ContainerDied","Data":"599c7ddcd2cc59591e83d43527ba030ab9e211c4a30960ca0bd7f18ba3839817"}
Jul 16 11:05:28.070525 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:05:28.070119    2583 scope.go:117] "RemoveContainer" containerID="f91fe19cb741d71812d03323b4d3a8becb368d45721a269ff996013f305cc9ba"
Jul 16 11:05:28.079307 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:05:28.079285    2583 scope.go:117] "RemoveContainer" containerID="f91fe19cb741d71812d03323b4d3a8becb368d45721a269ff996013f305cc9ba"
Jul 16 11:05:28.079584 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:05:28.079561    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f91fe19cb741d71812d03323b4d3a8becb368d45721a269ff996013f305cc9ba\": container with ID starting with f91fe19cb741d71812d03323b4d3a8becb368d45721a269ff996013f305cc9ba not found: ID does not exist" containerID="f91fe19cb741d71812d03323b4d3a8becb368d45721a269ff996013f305cc9ba"
Jul 16 11:05:28.079664 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:05:28.079609    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f91fe19cb741d71812d03323b4d3a8becb368d45721a269ff996013f305cc9ba"} err="failed to get container status \"f91fe19cb741d71812d03323b4d3a8becb368d45721a269ff996013f305cc9ba\": rpc error: code = NotFound desc = could not find container \"f91fe19cb741d71812d03323b4d3a8becb368d45721a269ff996013f305cc9ba\": container with ID starting with f91fe19cb741d71812d03323b4d3a8becb368d45721a269ff996013f305cc9ba not found: ID does not exist"
Jul 16 11:05:28.098807 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:05:28.098769    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-78b57b55c6-blcck"]
Jul 16 11:05:28.107577 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:05:28.107550    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-78b57b55c6-blcck"]
Jul 16 11:05:28.566005 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:05:28.565964    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-65f68df599-zw6jf" podUID="6a4b5dde-f664-4d97-889e-bfe4d4df1416" containerName="registry" containerID="cri-o://c8b43408320a38689e588fb042ef072b432fc519b8fa8934f8fcb796cd12d7d6" gracePeriod=30
Jul 16 11:05:28.841677 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:05:28.841581    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-65f68df599-zw6jf"
Jul 16 11:05:28.968104 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:05:28.968059    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/6a4b5dde-f664-4d97-889e-bfe4d4df1416-ca-trust-extracted\") pod \"6a4b5dde-f664-4d97-889e-bfe4d4df1416\" (UID: \"6a4b5dde-f664-4d97-889e-bfe4d4df1416\") "
Jul 16 11:05:28.968104 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:05:28.968104    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6a4b5dde-f664-4d97-889e-bfe4d4df1416-bound-sa-token\") pod \"6a4b5dde-f664-4d97-889e-bfe4d4df1416\" (UID: \"6a4b5dde-f664-4d97-889e-bfe4d4df1416\") "
Jul 16 11:05:28.968320 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:05:28.968134    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/6a4b5dde-f664-4d97-889e-bfe4d4df1416-registry-certificates\") pod \"6a4b5dde-f664-4d97-889e-bfe4d4df1416\" (UID: \"6a4b5dde-f664-4d97-889e-bfe4d4df1416\") "
Jul 16 11:05:28.968320 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:05:28.968168    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/6a4b5dde-f664-4d97-889e-bfe4d4df1416-installation-pull-secrets\") pod \"6a4b5dde-f664-4d97-889e-bfe4d4df1416\" (UID: \"6a4b5dde-f664-4d97-889e-bfe4d4df1416\") "
Jul 16 11:05:28.968320 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:05:28.968232    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/6a4b5dde-f664-4d97-889e-bfe4d4df1416-image-registry-private-configuration\") pod \"6a4b5dde-f664-4d97-889e-bfe4d4df1416\" (UID: \"6a4b5dde-f664-4d97-889e-bfe4d4df1416\") "
Jul 16 11:05:28.968320 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:05:28.968274    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6a4b5dde-f664-4d97-889e-bfe4d4df1416-registry-tls\") pod \"6a4b5dde-f664-4d97-889e-bfe4d4df1416\" (UID: \"6a4b5dde-f664-4d97-889e-bfe4d4df1416\") "
Jul 16 11:05:28.968320 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:05:28.968311    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6a4b5dde-f664-4d97-889e-bfe4d4df1416-trusted-ca\") pod \"6a4b5dde-f664-4d97-889e-bfe4d4df1416\" (UID: \"6a4b5dde-f664-4d97-889e-bfe4d4df1416\") "
Jul 16 11:05:28.968568 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:05:28.968360    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2mxl4\" (UniqueName: \"kubernetes.io/projected/6a4b5dde-f664-4d97-889e-bfe4d4df1416-kube-api-access-2mxl4\") pod \"6a4b5dde-f664-4d97-889e-bfe4d4df1416\" (UID: \"6a4b5dde-f664-4d97-889e-bfe4d4df1416\") "
Jul 16 11:05:28.969172 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:05:28.968680    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6a4b5dde-f664-4d97-889e-bfe4d4df1416-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "6a4b5dde-f664-4d97-889e-bfe4d4df1416" (UID: "6a4b5dde-f664-4d97-889e-bfe4d4df1416"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 11:05:28.969172 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:05:28.969068    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6a4b5dde-f664-4d97-889e-bfe4d4df1416-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "6a4b5dde-f664-4d97-889e-bfe4d4df1416" (UID: "6a4b5dde-f664-4d97-889e-bfe4d4df1416"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 11:05:28.970963 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:05:28.970895    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6a4b5dde-f664-4d97-889e-bfe4d4df1416-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "6a4b5dde-f664-4d97-889e-bfe4d4df1416" (UID: "6a4b5dde-f664-4d97-889e-bfe4d4df1416"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 11:05:28.971078 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:05:28.970991    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6a4b5dde-f664-4d97-889e-bfe4d4df1416-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "6a4b5dde-f664-4d97-889e-bfe4d4df1416" (UID: "6a4b5dde-f664-4d97-889e-bfe4d4df1416"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 11:05:28.971078 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:05:28.971061    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6a4b5dde-f664-4d97-889e-bfe4d4df1416-kube-api-access-2mxl4" (OuterVolumeSpecName: "kube-api-access-2mxl4") pod "6a4b5dde-f664-4d97-889e-bfe4d4df1416" (UID: "6a4b5dde-f664-4d97-889e-bfe4d4df1416"). InnerVolumeSpecName "kube-api-access-2mxl4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 11:05:28.971190 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:05:28.971077    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6a4b5dde-f664-4d97-889e-bfe4d4df1416-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "6a4b5dde-f664-4d97-889e-bfe4d4df1416" (UID: "6a4b5dde-f664-4d97-889e-bfe4d4df1416"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 11:05:28.971190 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:05:28.971113    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6a4b5dde-f664-4d97-889e-bfe4d4df1416-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "6a4b5dde-f664-4d97-889e-bfe4d4df1416" (UID: "6a4b5dde-f664-4d97-889e-bfe4d4df1416"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 11:05:28.976669 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:05:28.976639    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a4b5dde-f664-4d97-889e-bfe4d4df1416-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "6a4b5dde-f664-4d97-889e-bfe4d4df1416" (UID: "6a4b5dde-f664-4d97-889e-bfe4d4df1416"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 11:05:29.069676 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:05:29.069642    2583 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/6a4b5dde-f664-4d97-889e-bfe4d4df1416-registry-certificates\") on node \"ip-10-0-129-28.ec2.internal\" DevicePath \"\""
Jul 16 11:05:29.069676 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:05:29.069670    2583 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/6a4b5dde-f664-4d97-889e-bfe4d4df1416-installation-pull-secrets\") on node \"ip-10-0-129-28.ec2.internal\" DevicePath \"\""
Jul 16 11:05:29.069676 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:05:29.069681    2583 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/6a4b5dde-f664-4d97-889e-bfe4d4df1416-image-registry-private-configuration\") on node \"ip-10-0-129-28.ec2.internal\" DevicePath \"\""
Jul 16 11:05:29.069911 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:05:29.069691    2583 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6a4b5dde-f664-4d97-889e-bfe4d4df1416-registry-tls\") on node \"ip-10-0-129-28.ec2.internal\" DevicePath \"\""
Jul 16 11:05:29.069911 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:05:29.069699    2583 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6a4b5dde-f664-4d97-889e-bfe4d4df1416-trusted-ca\") on node \"ip-10-0-129-28.ec2.internal\" DevicePath \"\""
Jul 16 11:05:29.069911 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:05:29.069706    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2mxl4\" (UniqueName: \"kubernetes.io/projected/6a4b5dde-f664-4d97-889e-bfe4d4df1416-kube-api-access-2mxl4\") on node \"ip-10-0-129-28.ec2.internal\" DevicePath \"\""
Jul 16 11:05:29.069911 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:05:29.069714    2583 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/6a4b5dde-f664-4d97-889e-bfe4d4df1416-ca-trust-extracted\") on node \"ip-10-0-129-28.ec2.internal\" DevicePath \"\""
Jul 16 11:05:29.069911 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:05:29.069723    2583 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6a4b5dde-f664-4d97-889e-bfe4d4df1416-bound-sa-token\") on node \"ip-10-0-129-28.ec2.internal\" DevicePath \"\""
Jul 16 11:05:29.075641 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:05:29.075615    2583 generic.go:356] "Generic (PLEG): container finished" podID="6a4b5dde-f664-4d97-889e-bfe4d4df1416" containerID="c8b43408320a38689e588fb042ef072b432fc519b8fa8934f8fcb796cd12d7d6" exitCode=0
Jul 16 11:05:29.075782 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:05:29.075664    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-65f68df599-zw6jf"
Jul 16 11:05:29.075782 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:05:29.075684    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-65f68df599-zw6jf" event={"ID":"6a4b5dde-f664-4d97-889e-bfe4d4df1416","Type":"ContainerDied","Data":"c8b43408320a38689e588fb042ef072b432fc519b8fa8934f8fcb796cd12d7d6"}
Jul 16 11:05:29.075782 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:05:29.075735    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-65f68df599-zw6jf" event={"ID":"6a4b5dde-f664-4d97-889e-bfe4d4df1416","Type":"ContainerDied","Data":"40445f85624a9a6668acba8c6f028ab2465fc00cbb1dbffb0c9f2afb65d2790c"}
Jul 16 11:05:29.075782 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:05:29.075760    2583 scope.go:117] "RemoveContainer" containerID="c8b43408320a38689e588fb042ef072b432fc519b8fa8934f8fcb796cd12d7d6"
Jul 16 11:05:29.084845 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:05:29.084823    2583 scope.go:117] "RemoveContainer" containerID="c8b43408320a38689e588fb042ef072b432fc519b8fa8934f8fcb796cd12d7d6"
Jul 16 11:05:29.085117 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:05:29.085096    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c8b43408320a38689e588fb042ef072b432fc519b8fa8934f8fcb796cd12d7d6\": container with ID starting with c8b43408320a38689e588fb042ef072b432fc519b8fa8934f8fcb796cd12d7d6 not found: ID does not exist" containerID="c8b43408320a38689e588fb042ef072b432fc519b8fa8934f8fcb796cd12d7d6"
Jul 16 11:05:29.085169 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:05:29.085126    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c8b43408320a38689e588fb042ef072b432fc519b8fa8934f8fcb796cd12d7d6"} err="failed to get container status \"c8b43408320a38689e588fb042ef072b432fc519b8fa8934f8fcb796cd12d7d6\": rpc error: code = NotFound desc = could not find container \"c8b43408320a38689e588fb042ef072b432fc519b8fa8934f8fcb796cd12d7d6\": container with ID starting with c8b43408320a38689e588fb042ef072b432fc519b8fa8934f8fcb796cd12d7d6 not found: ID does not exist"
Jul 16 11:05:29.107120 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:05:29.107087    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-65f68df599-zw6jf"]
Jul 16 11:05:29.117566 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:05:29.117538    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-65f68df599-zw6jf"]
Jul 16 11:05:29.233305 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:05:29.233270    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6a4b5dde-f664-4d97-889e-bfe4d4df1416" path="/var/lib/kubelet/pods/6a4b5dde-f664-4d97-889e-bfe4d4df1416/volumes"
Jul 16 11:05:29.233748 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:05:29.233728    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cdd147a4-74f1-4ba7-a9dc-527960187ddc" path="/var/lib/kubelet/pods/cdd147a4-74f1-4ba7-a9dc-527960187ddc/volumes"
Jul 16 11:05:31.085701 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:05:31.085666    2583 generic.go:356] "Generic (PLEG): container finished" podID="543513a9-883b-4f90-8478-eb51b3186af1" containerID="cd09290e5f7521325b0cbd28367a0d41ea2da2fc67abc6b8a3582e4a96eee4bd" exitCode=0
Jul 16 11:05:31.086135 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:05:31.085720    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-7gl88" event={"ID":"543513a9-883b-4f90-8478-eb51b3186af1","Type":"ContainerDied","Data":"cd09290e5f7521325b0cbd28367a0d41ea2da2fc67abc6b8a3582e4a96eee4bd"}
Jul 16 11:05:31.086135 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:05:31.086003    2583 scope.go:117] "RemoveContainer" containerID="cd09290e5f7521325b0cbd28367a0d41ea2da2fc67abc6b8a3582e4a96eee4bd"
Jul 16 11:05:32.091155 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:05:32.091121    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-7gl88" event={"ID":"543513a9-883b-4f90-8478-eb51b3186af1","Type":"ContainerStarted","Data":"e9db99848d6eee5eb2eaac1360f06e7c14cc26512893a77cf218230cd73df8db"}
Jul 16 11:05:46.146491 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:05:46.146458    2583 generic.go:356] "Generic (PLEG): container finished" podID="e1f38c54-19be-4f6f-872b-39093678a7ad" containerID="e1e94a4013b98dabc8afaf41665a18c0bda154aeb7fa382747e18afa75c8e0bd" exitCode=0
Jul 16 11:05:46.146922 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:05:46.146516    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-85d88" event={"ID":"e1f38c54-19be-4f6f-872b-39093678a7ad","Type":"ContainerDied","Data":"e1e94a4013b98dabc8afaf41665a18c0bda154aeb7fa382747e18afa75c8e0bd"}
Jul 16 11:05:46.146922 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:05:46.146848    2583 scope.go:117] "RemoveContainer" containerID="e1e94a4013b98dabc8afaf41665a18c0bda154aeb7fa382747e18afa75c8e0bd"
Jul 16 11:05:47.152397 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:05:47.152357    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-85d88" event={"ID":"e1f38c54-19be-4f6f-872b-39093678a7ad","Type":"ContainerStarted","Data":"1eeb849f97983a668a3589b84cb85e8b932ce8022ecd020f117e3770f9dff157"}
Jul 16 11:06:24.340187 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:06:24.340113    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-85b59797cc-4n9bf"]
Jul 16 11:06:24.340658 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:06:24.340414    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cdd147a4-74f1-4ba7-a9dc-527960187ddc" containerName="console"
Jul 16 11:06:24.340658 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:06:24.340426    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="cdd147a4-74f1-4ba7-a9dc-527960187ddc" containerName="console"
Jul 16 11:06:24.340658 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:06:24.340453    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6a4b5dde-f664-4d97-889e-bfe4d4df1416" containerName="registry"
Jul 16 11:06:24.340658 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:06:24.340459    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="6a4b5dde-f664-4d97-889e-bfe4d4df1416" containerName="registry"
Jul 16 11:06:24.340658 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:06:24.340514    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="6a4b5dde-f664-4d97-889e-bfe4d4df1416" containerName="registry"
Jul 16 11:06:24.340658 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:06:24.340521    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="cdd147a4-74f1-4ba7-a9dc-527960187ddc" containerName="console"
Jul 16 11:06:24.342511 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:06:24.342487    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-85b59797cc-4n9bf"
Jul 16 11:06:24.345298 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:06:24.345280    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 16 11:06:24.345298 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:06:24.345288    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 16 11:06:24.346566 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:06:24.346407    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-bst9c\""
Jul 16 11:06:24.346566 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:06:24.346453    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 16 11:06:24.346566 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:06:24.346505    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 16 11:06:24.346566 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:06:24.346511    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 16 11:06:24.351558 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:06:24.351531    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 16 11:06:24.356264 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:06:24.356221    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-85b59797cc-4n9bf"]
Jul 16 11:06:24.363403 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:06:24.363370    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/694ac9ec-660f-46b9-87d8-49aa9b500d89-trusted-ca-bundle\") pod \"console-85b59797cc-4n9bf\" (UID: \"694ac9ec-660f-46b9-87d8-49aa9b500d89\") " pod="openshift-console/console-85b59797cc-4n9bf"
Jul 16 11:06:24.363403 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:06:24.363406    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zqxw7\" (UniqueName: \"kubernetes.io/projected/694ac9ec-660f-46b9-87d8-49aa9b500d89-kube-api-access-zqxw7\") pod \"console-85b59797cc-4n9bf\" (UID: \"694ac9ec-660f-46b9-87d8-49aa9b500d89\") " pod="openshift-console/console-85b59797cc-4n9bf"
Jul 16 11:06:24.363636 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:06:24.363436    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/694ac9ec-660f-46b9-87d8-49aa9b500d89-oauth-serving-cert\") pod \"console-85b59797cc-4n9bf\" (UID: \"694ac9ec-660f-46b9-87d8-49aa9b500d89\") " pod="openshift-console/console-85b59797cc-4n9bf"
Jul 16 11:06:24.363636 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:06:24.363544    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/694ac9ec-660f-46b9-87d8-49aa9b500d89-console-oauth-config\") pod \"console-85b59797cc-4n9bf\" (UID: \"694ac9ec-660f-46b9-87d8-49aa9b500d89\") " pod="openshift-console/console-85b59797cc-4n9bf"
Jul 16 11:06:24.363636 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:06:24.363615    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/694ac9ec-660f-46b9-87d8-49aa9b500d89-console-serving-cert\") pod \"console-85b59797cc-4n9bf\" (UID: \"694ac9ec-660f-46b9-87d8-49aa9b500d89\") " pod="openshift-console/console-85b59797cc-4n9bf"
Jul 16 11:06:24.363755 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:06:24.363653    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/694ac9ec-660f-46b9-87d8-49aa9b500d89-console-config\") pod \"console-85b59797cc-4n9bf\" (UID: \"694ac9ec-660f-46b9-87d8-49aa9b500d89\") " pod="openshift-console/console-85b59797cc-4n9bf"
Jul 16 11:06:24.363755 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:06:24.363686    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/694ac9ec-660f-46b9-87d8-49aa9b500d89-service-ca\") pod \"console-85b59797cc-4n9bf\" (UID: \"694ac9ec-660f-46b9-87d8-49aa9b500d89\") " pod="openshift-console/console-85b59797cc-4n9bf"
Jul 16 11:06:24.464348 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:06:24.464301    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/694ac9ec-660f-46b9-87d8-49aa9b500d89-console-serving-cert\") pod \"console-85b59797cc-4n9bf\" (UID: \"694ac9ec-660f-46b9-87d8-49aa9b500d89\") " pod="openshift-console/console-85b59797cc-4n9bf"
Jul 16 11:06:24.464348 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:06:24.464359    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/694ac9ec-660f-46b9-87d8-49aa9b500d89-console-config\") pod \"console-85b59797cc-4n9bf\" (UID: \"694ac9ec-660f-46b9-87d8-49aa9b500d89\") " pod="openshift-console/console-85b59797cc-4n9bf"
Jul 16 11:06:24.464633 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:06:24.464386    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/694ac9ec-660f-46b9-87d8-49aa9b500d89-service-ca\") pod \"console-85b59797cc-4n9bf\" (UID: \"694ac9ec-660f-46b9-87d8-49aa9b500d89\") " pod="openshift-console/console-85b59797cc-4n9bf"
Jul 16 11:06:24.464633 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:06:24.464444    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/694ac9ec-660f-46b9-87d8-49aa9b500d89-trusted-ca-bundle\") pod \"console-85b59797cc-4n9bf\" (UID: \"694ac9ec-660f-46b9-87d8-49aa9b500d89\") " pod="openshift-console/console-85b59797cc-4n9bf"
Jul 16 11:06:24.464633 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:06:24.464468    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zqxw7\" (UniqueName: \"kubernetes.io/projected/694ac9ec-660f-46b9-87d8-49aa9b500d89-kube-api-access-zqxw7\") pod \"console-85b59797cc-4n9bf\" (UID: \"694ac9ec-660f-46b9-87d8-49aa9b500d89\") " pod="openshift-console/console-85b59797cc-4n9bf"
Jul 16 11:06:24.464633 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:06:24.464505    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/694ac9ec-660f-46b9-87d8-49aa9b500d89-oauth-serving-cert\") pod \"console-85b59797cc-4n9bf\" (UID: \"694ac9ec-660f-46b9-87d8-49aa9b500d89\") " pod="openshift-console/console-85b59797cc-4n9bf"
Jul 16 11:06:24.464633 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:06:24.464554    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/694ac9ec-660f-46b9-87d8-49aa9b500d89-console-oauth-config\") pod \"console-85b59797cc-4n9bf\" (UID: \"694ac9ec-660f-46b9-87d8-49aa9b500d89\") " pod="openshift-console/console-85b59797cc-4n9bf"
Jul 16 11:06:24.465386 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:06:24.465354    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/694ac9ec-660f-46b9-87d8-49aa9b500d89-console-config\") pod \"console-85b59797cc-4n9bf\" (UID: \"694ac9ec-660f-46b9-87d8-49aa9b500d89\") " pod="openshift-console/console-85b59797cc-4n9bf"
Jul 16 11:06:24.465519 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:06:24.465396    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/694ac9ec-660f-46b9-87d8-49aa9b500d89-oauth-serving-cert\") pod \"console-85b59797cc-4n9bf\" (UID: \"694ac9ec-660f-46b9-87d8-49aa9b500d89\") " pod="openshift-console/console-85b59797cc-4n9bf"
Jul 16 11:06:24.465519 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:06:24.465440    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/694ac9ec-660f-46b9-87d8-49aa9b500d89-trusted-ca-bundle\") pod \"console-85b59797cc-4n9bf\" (UID: \"694ac9ec-660f-46b9-87d8-49aa9b500d89\") " pod="openshift-console/console-85b59797cc-4n9bf"
Jul 16 11:06:24.465519 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:06:24.465438    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/694ac9ec-660f-46b9-87d8-49aa9b500d89-service-ca\") pod \"console-85b59797cc-4n9bf\" (UID: \"694ac9ec-660f-46b9-87d8-49aa9b500d89\") " pod="openshift-console/console-85b59797cc-4n9bf"
Jul 16 11:06:24.467093 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:06:24.467058    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/694ac9ec-660f-46b9-87d8-49aa9b500d89-console-oauth-config\") pod \"console-85b59797cc-4n9bf\" (UID: \"694ac9ec-660f-46b9-87d8-49aa9b500d89\") " pod="openshift-console/console-85b59797cc-4n9bf"
Jul 16 11:06:24.467198 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:06:24.467141    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/694ac9ec-660f-46b9-87d8-49aa9b500d89-console-serving-cert\") pod \"console-85b59797cc-4n9bf\" (UID: \"694ac9ec-660f-46b9-87d8-49aa9b500d89\") " pod="openshift-console/console-85b59797cc-4n9bf"
Jul 16 11:06:24.473146 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:06:24.473117    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zqxw7\" (UniqueName: \"kubernetes.io/projected/694ac9ec-660f-46b9-87d8-49aa9b500d89-kube-api-access-zqxw7\") pod \"console-85b59797cc-4n9bf\" (UID: \"694ac9ec-660f-46b9-87d8-49aa9b500d89\") " pod="openshift-console/console-85b59797cc-4n9bf"
Jul 16 11:06:24.655355 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:06:24.655318    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-85b59797cc-4n9bf"
Jul 16 11:06:24.797412 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:06:24.797386    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-85b59797cc-4n9bf"]
Jul 16 11:06:24.800135 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:06:24.800101    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod694ac9ec_660f_46b9_87d8_49aa9b500d89.slice/crio-9cae1986708bf34ce446e1b576d2536a9bb842a97b9f792f2a7740e340d81840 WatchSource:0}: Error finding container 9cae1986708bf34ce446e1b576d2536a9bb842a97b9f792f2a7740e340d81840: Status 404 returned error can't find the container with id 9cae1986708bf34ce446e1b576d2536a9bb842a97b9f792f2a7740e340d81840
Jul 16 11:06:25.295875 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:06:25.295836    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-85b59797cc-4n9bf" event={"ID":"694ac9ec-660f-46b9-87d8-49aa9b500d89","Type":"ContainerStarted","Data":"7e8e54f7fe4166a0d06277aa3e4d847d1276e5c9b2f62b002717fe762d62421b"}
Jul 16 11:06:25.296068 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:06:25.295892    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-85b59797cc-4n9bf" event={"ID":"694ac9ec-660f-46b9-87d8-49aa9b500d89","Type":"ContainerStarted","Data":"9cae1986708bf34ce446e1b576d2536a9bb842a97b9f792f2a7740e340d81840"}
Jul 16 11:06:25.321467 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:06:25.321412    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-85b59797cc-4n9bf" podStartSLOduration=1.321395682 podStartE2EDuration="1.321395682s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 11:06:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 11:06:25.320236461 +0000 UTC m=+177.954457917" watchObservedRunningTime="2026-07-16 11:06:25.321395682 +0000 UTC m=+177.955617135"
Jul 16 11:06:33.376504 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:06:33.376464    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-85b59797cc-4n9bf"]
Jul 16 11:06:33.409868 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:06:33.409838    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-cb66496d-mfk4x"]
Jul 16 11:06:33.413056 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:06:33.413035    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-cb66496d-mfk4x"
Jul 16 11:06:33.425497 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:06:33.425468    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-cb66496d-mfk4x"]
Jul 16 11:06:33.437294 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:06:33.437256    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/335cacdf-3743-4030-ac8f-66723698cab1-console-config\") pod \"console-cb66496d-mfk4x\" (UID: \"335cacdf-3743-4030-ac8f-66723698cab1\") " pod="openshift-console/console-cb66496d-mfk4x"
Jul 16 11:06:33.437435 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:06:33.437302    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6g9sc\" (UniqueName: \"kubernetes.io/projected/335cacdf-3743-4030-ac8f-66723698cab1-kube-api-access-6g9sc\") pod \"console-cb66496d-mfk4x\" (UID: \"335cacdf-3743-4030-ac8f-66723698cab1\") " pod="openshift-console/console-cb66496d-mfk4x"
Jul 16 11:06:33.437435 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:06:33.437384    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/335cacdf-3743-4030-ac8f-66723698cab1-console-oauth-config\") pod \"console-cb66496d-mfk4x\" (UID: \"335cacdf-3743-4030-ac8f-66723698cab1\") " pod="openshift-console/console-cb66496d-mfk4x"
Jul 16 11:06:33.437435 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:06:33.437428    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/335cacdf-3743-4030-ac8f-66723698cab1-console-serving-cert\") pod \"console-cb66496d-mfk4x\" (UID: \"335cacdf-3743-4030-ac8f-66723698cab1\") " pod="openshift-console/console-cb66496d-mfk4x"
Jul 16 11:06:33.437554 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:06:33.437448    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/335cacdf-3743-4030-ac8f-66723698cab1-trusted-ca-bundle\") pod \"console-cb66496d-mfk4x\" (UID: \"335cacdf-3743-4030-ac8f-66723698cab1\") " pod="openshift-console/console-cb66496d-mfk4x"
Jul 16 11:06:33.437554 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:06:33.437485    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/335cacdf-3743-4030-ac8f-66723698cab1-service-ca\") pod \"console-cb66496d-mfk4x\" (UID: \"335cacdf-3743-4030-ac8f-66723698cab1\") " pod="openshift-console/console-cb66496d-mfk4x"
Jul 16 11:06:33.437554 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:06:33.437552    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/335cacdf-3743-4030-ac8f-66723698cab1-oauth-serving-cert\") pod \"console-cb66496d-mfk4x\" (UID: \"335cacdf-3743-4030-ac8f-66723698cab1\") " pod="openshift-console/console-cb66496d-mfk4x"
Jul 16 11:06:33.538529 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:06:33.538478    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/335cacdf-3743-4030-ac8f-66723698cab1-oauth-serving-cert\") pod \"console-cb66496d-mfk4x\" (UID: \"335cacdf-3743-4030-ac8f-66723698cab1\") " pod="openshift-console/console-cb66496d-mfk4x"
Jul 16 11:06:33.538529 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:06:33.538530    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/335cacdf-3743-4030-ac8f-66723698cab1-console-config\") pod \"console-cb66496d-mfk4x\" (UID: \"335cacdf-3743-4030-ac8f-66723698cab1\") " pod="openshift-console/console-cb66496d-mfk4x"
Jul 16 11:06:33.538810 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:06:33.538547    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6g9sc\" (UniqueName: \"kubernetes.io/projected/335cacdf-3743-4030-ac8f-66723698cab1-kube-api-access-6g9sc\") pod \"console-cb66496d-mfk4x\" (UID: \"335cacdf-3743-4030-ac8f-66723698cab1\") " pod="openshift-console/console-cb66496d-mfk4x"
Jul 16 11:06:33.538810 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:06:33.538588    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/335cacdf-3743-4030-ac8f-66723698cab1-console-oauth-config\") pod \"console-cb66496d-mfk4x\" (UID: \"335cacdf-3743-4030-ac8f-66723698cab1\") " pod="openshift-console/console-cb66496d-mfk4x"
Jul 16 11:06:33.538810 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:06:33.538642    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/335cacdf-3743-4030-ac8f-66723698cab1-console-serving-cert\") pod \"console-cb66496d-mfk4x\" (UID: \"335cacdf-3743-4030-ac8f-66723698cab1\") " pod="openshift-console/console-cb66496d-mfk4x"
Jul 16 11:06:33.538810 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:06:33.538665    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/335cacdf-3743-4030-ac8f-66723698cab1-trusted-ca-bundle\") pod \"console-cb66496d-mfk4x\" (UID: \"335cacdf-3743-4030-ac8f-66723698cab1\") " pod="openshift-console/console-cb66496d-mfk4x"
Jul 16 11:06:33.538810 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:06:33.538695    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/335cacdf-3743-4030-ac8f-66723698cab1-service-ca\") pod \"console-cb66496d-mfk4x\" (UID: \"335cacdf-3743-4030-ac8f-66723698cab1\") " pod="openshift-console/console-cb66496d-mfk4x"
Jul 16 11:06:33.539441 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:06:33.539413    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/335cacdf-3743-4030-ac8f-66723698cab1-service-ca\") pod \"console-cb66496d-mfk4x\" (UID: \"335cacdf-3743-4030-ac8f-66723698cab1\") " pod="openshift-console/console-cb66496d-mfk4x"
Jul 16 11:06:33.539549 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:06:33.539413    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/335cacdf-3743-4030-ac8f-66723698cab1-console-config\") pod \"console-cb66496d-mfk4x\" (UID: \"335cacdf-3743-4030-ac8f-66723698cab1\") " pod="openshift-console/console-cb66496d-mfk4x"
Jul 16 11:06:33.539549 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:06:33.539477    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/335cacdf-3743-4030-ac8f-66723698cab1-oauth-serving-cert\") pod \"console-cb66496d-mfk4x\" (UID: \"335cacdf-3743-4030-ac8f-66723698cab1\") " pod="openshift-console/console-cb66496d-mfk4x"
Jul 16 11:06:33.539761 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:06:33.539555    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/335cacdf-3743-4030-ac8f-66723698cab1-trusted-ca-bundle\") pod \"console-cb66496d-mfk4x\" (UID: \"335cacdf-3743-4030-ac8f-66723698cab1\") " pod="openshift-console/console-cb66496d-mfk4x"
Jul 16 11:06:33.541135 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:06:33.541098    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/335cacdf-3743-4030-ac8f-66723698cab1-console-oauth-config\") pod \"console-cb66496d-mfk4x\" (UID: \"335cacdf-3743-4030-ac8f-66723698cab1\") " pod="openshift-console/console-cb66496d-mfk4x"
Jul 16 11:06:33.541264 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:06:33.541238    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/335cacdf-3743-4030-ac8f-66723698cab1-console-serving-cert\") pod \"console-cb66496d-mfk4x\" (UID: \"335cacdf-3743-4030-ac8f-66723698cab1\") " pod="openshift-console/console-cb66496d-mfk4x"
Jul 16 11:06:33.549019 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:06:33.548995    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6g9sc\" (UniqueName: \"kubernetes.io/projected/335cacdf-3743-4030-ac8f-66723698cab1-kube-api-access-6g9sc\") pod \"console-cb66496d-mfk4x\" (UID: \"335cacdf-3743-4030-ac8f-66723698cab1\") " pod="openshift-console/console-cb66496d-mfk4x"
Jul 16 11:06:33.722772 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:06:33.722736    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-cb66496d-mfk4x"
Jul 16 11:06:33.853801 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:06:33.853774    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-cb66496d-mfk4x"]
Jul 16 11:06:33.856369 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:06:33.856342    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod335cacdf_3743_4030_ac8f_66723698cab1.slice/crio-18c3e782fedadddf3eb5118022cf26c8bd3852715a31b318ce8dc67cc4036e78 WatchSource:0}: Error finding container 18c3e782fedadddf3eb5118022cf26c8bd3852715a31b318ce8dc67cc4036e78: Status 404 returned error can't find the container with id 18c3e782fedadddf3eb5118022cf26c8bd3852715a31b318ce8dc67cc4036e78
Jul 16 11:06:34.331751 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:06:34.331710    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-cb66496d-mfk4x" event={"ID":"335cacdf-3743-4030-ac8f-66723698cab1","Type":"ContainerStarted","Data":"ff93884e8de7374e84778327f4d195d03ea5cb51c1cfebfa9e062a7c285539c9"}
Jul 16 11:06:34.331955 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:06:34.331763    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-cb66496d-mfk4x" event={"ID":"335cacdf-3743-4030-ac8f-66723698cab1","Type":"ContainerStarted","Data":"18c3e782fedadddf3eb5118022cf26c8bd3852715a31b318ce8dc67cc4036e78"}
Jul 16 11:06:34.352706 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:06:34.352646    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-cb66496d-mfk4x" podStartSLOduration=1.352625562 podStartE2EDuration="1.352625562s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 11:06:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 11:06:34.352526588 +0000 UTC m=+186.986748069" watchObservedRunningTime="2026-07-16 11:06:34.352625562 +0000 UTC m=+186.986847017"
Jul 16 11:06:34.656088 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:06:34.656039    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-85b59797cc-4n9bf"
Jul 16 11:06:43.723109 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:06:43.723068    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-cb66496d-mfk4x"
Jul 16 11:06:43.723525 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:06:43.723150    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-cb66496d-mfk4x"
Jul 16 11:06:43.727970 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:06:43.727943    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-cb66496d-mfk4x"
Jul 16 11:06:44.375426 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:06:44.375399    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-cb66496d-mfk4x"
Jul 16 11:06:58.398320 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:06:58.398277    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-85b59797cc-4n9bf" podUID="694ac9ec-660f-46b9-87d8-49aa9b500d89" containerName="console" containerID="cri-o://7e8e54f7fe4166a0d06277aa3e4d847d1276e5c9b2f62b002717fe762d62421b" gracePeriod=15
Jul 16 11:06:58.648251 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:06:58.648225    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-85b59797cc-4n9bf"
Jul 16 11:06:58.754789 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:06:58.754693    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/694ac9ec-660f-46b9-87d8-49aa9b500d89-service-ca\") pod \"694ac9ec-660f-46b9-87d8-49aa9b500d89\" (UID: \"694ac9ec-660f-46b9-87d8-49aa9b500d89\") "
Jul 16 11:06:58.754789 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:06:58.754748    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zqxw7\" (UniqueName: \"kubernetes.io/projected/694ac9ec-660f-46b9-87d8-49aa9b500d89-kube-api-access-zqxw7\") pod \"694ac9ec-660f-46b9-87d8-49aa9b500d89\" (UID: \"694ac9ec-660f-46b9-87d8-49aa9b500d89\") "
Jul 16 11:06:58.755007 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:06:58.754793    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/694ac9ec-660f-46b9-87d8-49aa9b500d89-console-oauth-config\") pod \"694ac9ec-660f-46b9-87d8-49aa9b500d89\" (UID: \"694ac9ec-660f-46b9-87d8-49aa9b500d89\") "
Jul 16 11:06:58.755007 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:06:58.754814    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/694ac9ec-660f-46b9-87d8-49aa9b500d89-console-config\") pod \"694ac9ec-660f-46b9-87d8-49aa9b500d89\" (UID: \"694ac9ec-660f-46b9-87d8-49aa9b500d89\") "
Jul 16 11:06:58.755007 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:06:58.754843    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/694ac9ec-660f-46b9-87d8-49aa9b500d89-trusted-ca-bundle\") pod \"694ac9ec-660f-46b9-87d8-49aa9b500d89\" (UID: \"694ac9ec-660f-46b9-87d8-49aa9b500d89\") "
Jul 16 11:06:58.755007 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:06:58.754865    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/694ac9ec-660f-46b9-87d8-49aa9b500d89-oauth-serving-cert\") pod \"694ac9ec-660f-46b9-87d8-49aa9b500d89\" (UID: \"694ac9ec-660f-46b9-87d8-49aa9b500d89\") "
Jul 16 11:06:58.755007 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:06:58.754950    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/694ac9ec-660f-46b9-87d8-49aa9b500d89-console-serving-cert\") pod \"694ac9ec-660f-46b9-87d8-49aa9b500d89\" (UID: \"694ac9ec-660f-46b9-87d8-49aa9b500d89\") "
Jul 16 11:06:58.755264 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:06:58.755233    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/694ac9ec-660f-46b9-87d8-49aa9b500d89-service-ca" (OuterVolumeSpecName: "service-ca") pod "694ac9ec-660f-46b9-87d8-49aa9b500d89" (UID: "694ac9ec-660f-46b9-87d8-49aa9b500d89"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 11:06:58.755332 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:06:58.755303    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/694ac9ec-660f-46b9-87d8-49aa9b500d89-console-config" (OuterVolumeSpecName: "console-config") pod "694ac9ec-660f-46b9-87d8-49aa9b500d89" (UID: "694ac9ec-660f-46b9-87d8-49aa9b500d89"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 11:06:58.755395 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:06:58.755306    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/694ac9ec-660f-46b9-87d8-49aa9b500d89-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "694ac9ec-660f-46b9-87d8-49aa9b500d89" (UID: "694ac9ec-660f-46b9-87d8-49aa9b500d89"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 11:06:58.755395 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:06:58.755311    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/694ac9ec-660f-46b9-87d8-49aa9b500d89-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "694ac9ec-660f-46b9-87d8-49aa9b500d89" (UID: "694ac9ec-660f-46b9-87d8-49aa9b500d89"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 11:06:58.757117 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:06:58.757090    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/694ac9ec-660f-46b9-87d8-49aa9b500d89-kube-api-access-zqxw7" (OuterVolumeSpecName: "kube-api-access-zqxw7") pod "694ac9ec-660f-46b9-87d8-49aa9b500d89" (UID: "694ac9ec-660f-46b9-87d8-49aa9b500d89"). InnerVolumeSpecName "kube-api-access-zqxw7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 11:06:58.757117 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:06:58.757107    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/694ac9ec-660f-46b9-87d8-49aa9b500d89-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "694ac9ec-660f-46b9-87d8-49aa9b500d89" (UID: "694ac9ec-660f-46b9-87d8-49aa9b500d89"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 11:06:58.757273 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:06:58.757170    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/694ac9ec-660f-46b9-87d8-49aa9b500d89-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "694ac9ec-660f-46b9-87d8-49aa9b500d89" (UID: "694ac9ec-660f-46b9-87d8-49aa9b500d89"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 11:06:58.855608 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:06:58.855565    2583 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/694ac9ec-660f-46b9-87d8-49aa9b500d89-console-oauth-config\") on node \"ip-10-0-129-28.ec2.internal\" DevicePath \"\""
Jul 16 11:06:58.855772 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:06:58.855630    2583 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/694ac9ec-660f-46b9-87d8-49aa9b500d89-console-config\") on node \"ip-10-0-129-28.ec2.internal\" DevicePath \"\""
Jul 16 11:06:58.855772 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:06:58.855642    2583 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/694ac9ec-660f-46b9-87d8-49aa9b500d89-trusted-ca-bundle\") on node \"ip-10-0-129-28.ec2.internal\" DevicePath \"\""
Jul 16 11:06:58.855772 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:06:58.855651    2583 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/694ac9ec-660f-46b9-87d8-49aa9b500d89-oauth-serving-cert\") on node \"ip-10-0-129-28.ec2.internal\" DevicePath \"\""
Jul 16 11:06:58.855772 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:06:58.855659    2583 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/694ac9ec-660f-46b9-87d8-49aa9b500d89-console-serving-cert\") on node \"ip-10-0-129-28.ec2.internal\" DevicePath \"\""
Jul 16 11:06:58.855772 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:06:58.855668    2583 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/694ac9ec-660f-46b9-87d8-49aa9b500d89-service-ca\") on node \"ip-10-0-129-28.ec2.internal\" DevicePath \"\""
Jul 16 11:06:58.855772 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:06:58.855676    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zqxw7\" (UniqueName: \"kubernetes.io/projected/694ac9ec-660f-46b9-87d8-49aa9b500d89-kube-api-access-zqxw7\") on node \"ip-10-0-129-28.ec2.internal\" DevicePath \"\""
Jul 16 11:06:59.430173 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:06:59.430135    2583 generic.go:356] "Generic (PLEG): container finished" podID="694ac9ec-660f-46b9-87d8-49aa9b500d89" containerID="7e8e54f7fe4166a0d06277aa3e4d847d1276e5c9b2f62b002717fe762d62421b" exitCode=0
Jul 16 11:06:59.430554 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:06:59.430178    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-85b59797cc-4n9bf" event={"ID":"694ac9ec-660f-46b9-87d8-49aa9b500d89","Type":"ContainerDied","Data":"7e8e54f7fe4166a0d06277aa3e4d847d1276e5c9b2f62b002717fe762d62421b"}
Jul 16 11:06:59.430554 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:06:59.430219    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-85b59797cc-4n9bf" event={"ID":"694ac9ec-660f-46b9-87d8-49aa9b500d89","Type":"ContainerDied","Data":"9cae1986708bf34ce446e1b576d2536a9bb842a97b9f792f2a7740e340d81840"}
Jul 16 11:06:59.430554 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:06:59.430239    2583 scope.go:117] "RemoveContainer" containerID="7e8e54f7fe4166a0d06277aa3e4d847d1276e5c9b2f62b002717fe762d62421b"
Jul 16 11:06:59.430554 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:06:59.430178    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-85b59797cc-4n9bf"
Jul 16 11:06:59.438381 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:06:59.438363    2583 scope.go:117] "RemoveContainer" containerID="7e8e54f7fe4166a0d06277aa3e4d847d1276e5c9b2f62b002717fe762d62421b"
Jul 16 11:06:59.438653 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:06:59.438633    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7e8e54f7fe4166a0d06277aa3e4d847d1276e5c9b2f62b002717fe762d62421b\": container with ID starting with 7e8e54f7fe4166a0d06277aa3e4d847d1276e5c9b2f62b002717fe762d62421b not found: ID does not exist" containerID="7e8e54f7fe4166a0d06277aa3e4d847d1276e5c9b2f62b002717fe762d62421b"
Jul 16 11:06:59.438728 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:06:59.438668    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7e8e54f7fe4166a0d06277aa3e4d847d1276e5c9b2f62b002717fe762d62421b"} err="failed to get container status \"7e8e54f7fe4166a0d06277aa3e4d847d1276e5c9b2f62b002717fe762d62421b\": rpc error: code = NotFound desc = could not find container \"7e8e54f7fe4166a0d06277aa3e4d847d1276e5c9b2f62b002717fe762d62421b\": container with ID starting with 7e8e54f7fe4166a0d06277aa3e4d847d1276e5c9b2f62b002717fe762d62421b not found: ID does not exist"
Jul 16 11:06:59.450928 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:06:59.450899    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-85b59797cc-4n9bf"]
Jul 16 11:06:59.456625 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:06:59.456584    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-85b59797cc-4n9bf"]
Jul 16 11:07:01.228120 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:07:01.228075    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="694ac9ec-660f-46b9-87d8-49aa9b500d89" path="/var/lib/kubelet/pods/694ac9ec-660f-46b9-87d8-49aa9b500d89/volumes"
Jul 16 11:07:58.406034 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:07:58.405951    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-vq9dg"]
Jul 16 11:07:58.406477 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:07:58.406248    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="694ac9ec-660f-46b9-87d8-49aa9b500d89" containerName="console"
Jul 16 11:07:58.406477 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:07:58.406258    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="694ac9ec-660f-46b9-87d8-49aa9b500d89" containerName="console"
Jul 16 11:07:58.406477 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:07:58.406339    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="694ac9ec-660f-46b9-87d8-49aa9b500d89" containerName="console"
Jul 16 11:07:58.408284 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:07:58.408266    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-vq9dg"
Jul 16 11:07:58.410814 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:07:58.410782    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Jul 16 11:07:58.410814 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:07:58.410802    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Jul 16 11:07:58.410995 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:07:58.410850    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-9pjzq\""
Jul 16 11:07:58.423450 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:07:58.423403    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-vq9dg"]
Jul 16 11:07:58.523842 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:07:58.523799    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sqnlc\" (UniqueName: \"kubernetes.io/projected/b300fa60-3ab0-44e6-9878-8a8a8b0e985d-kube-api-access-sqnlc\") pod \"cert-manager-operator-controller-manager-55c444bf86-vq9dg\" (UID: \"b300fa60-3ab0-44e6-9878-8a8a8b0e985d\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-vq9dg"
Jul 16 11:07:58.523842 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:07:58.523845    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b300fa60-3ab0-44e6-9878-8a8a8b0e985d-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-vq9dg\" (UID: \"b300fa60-3ab0-44e6-9878-8a8a8b0e985d\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-vq9dg"
Jul 16 11:07:58.624504 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:07:58.624457    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sqnlc\" (UniqueName: \"kubernetes.io/projected/b300fa60-3ab0-44e6-9878-8a8a8b0e985d-kube-api-access-sqnlc\") pod \"cert-manager-operator-controller-manager-55c444bf86-vq9dg\" (UID: \"b300fa60-3ab0-44e6-9878-8a8a8b0e985d\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-vq9dg"
Jul 16 11:07:58.624504 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:07:58.624504    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b300fa60-3ab0-44e6-9878-8a8a8b0e985d-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-vq9dg\" (UID: \"b300fa60-3ab0-44e6-9878-8a8a8b0e985d\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-vq9dg"
Jul 16 11:07:58.624943 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:07:58.624926    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b300fa60-3ab0-44e6-9878-8a8a8b0e985d-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-vq9dg\" (UID: \"b300fa60-3ab0-44e6-9878-8a8a8b0e985d\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-vq9dg"
Jul 16 11:07:58.635921 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:07:58.635893    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sqnlc\" (UniqueName: \"kubernetes.io/projected/b300fa60-3ab0-44e6-9878-8a8a8b0e985d-kube-api-access-sqnlc\") pod \"cert-manager-operator-controller-manager-55c444bf86-vq9dg\" (UID: \"b300fa60-3ab0-44e6-9878-8a8a8b0e985d\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-vq9dg"
Jul 16 11:07:58.717764 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:07:58.717672    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-vq9dg"
Jul 16 11:07:58.860347 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:07:58.860327    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-vq9dg"]
Jul 16 11:07:58.862559 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:07:58.862533    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb300fa60_3ab0_44e6_9878_8a8a8b0e985d.slice/crio-8fe10c1116e6769f20b2582641b5f01c752d0a03a0d8e69ae8f16ec722e7492c WatchSource:0}: Error finding container 8fe10c1116e6769f20b2582641b5f01c752d0a03a0d8e69ae8f16ec722e7492c: Status 404 returned error can't find the container with id 8fe10c1116e6769f20b2582641b5f01c752d0a03a0d8e69ae8f16ec722e7492c
Jul 16 11:07:59.660302 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:07:59.660267    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-vq9dg" event={"ID":"b300fa60-3ab0-44e6-9878-8a8a8b0e985d","Type":"ContainerStarted","Data":"8fe10c1116e6769f20b2582641b5f01c752d0a03a0d8e69ae8f16ec722e7492c"}
Jul 16 11:08:02.676552 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:08:02.676511    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-vq9dg" event={"ID":"b300fa60-3ab0-44e6-9878-8a8a8b0e985d","Type":"ContainerStarted","Data":"1f812f9a2f72c1d2c510b55931e353f5e5f8a3fa0ebc43664afa60031f5d131a"}
Jul 16 11:08:02.691683 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:08:02.691650    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-xmqdc"]
Jul 16 11:08:02.693952 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:08:02.693934    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-xmqdc"
Jul 16 11:08:02.696368 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:08:02.696341    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 16 11:08:02.696368 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:08:02.696365    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 16 11:08:02.704702 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:08:02.704649    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-vq9dg" podStartSLOduration=1.949602752 podStartE2EDuration="4.704628836s" totalImagesPullingTime="2.755026084s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 11:07:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 11:08:02.702780478 +0000 UTC m=+275.337001944" watchObservedRunningTime="2026-07-16 11:08:02.704628836 +0000 UTC m=+275.338850293"
Jul 16 11:08:02.707757 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:08:02.707727    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-xmqdc"]
Jul 16 11:08:02.759345 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:08:02.759307    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ljswn\" (UniqueName: \"kubernetes.io/projected/4fe909e4-fe6e-4936-ad12-e6499db3eb61-kube-api-access-ljswn\") pod \"cert-manager-webhook-6486ffd997-xmqdc\" (UID: \"4fe909e4-fe6e-4936-ad12-e6499db3eb61\") " pod="cert-manager/cert-manager-webhook-6486ffd997-xmqdc"
Jul 16 11:08:02.759548 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:08:02.759436    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4fe909e4-fe6e-4936-ad12-e6499db3eb61-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-xmqdc\" (UID: \"4fe909e4-fe6e-4936-ad12-e6499db3eb61\") " pod="cert-manager/cert-manager-webhook-6486ffd997-xmqdc"
Jul 16 11:08:02.860633 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:08:02.860566    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ljswn\" (UniqueName: \"kubernetes.io/projected/4fe909e4-fe6e-4936-ad12-e6499db3eb61-kube-api-access-ljswn\") pod \"cert-manager-webhook-6486ffd997-xmqdc\" (UID: \"4fe909e4-fe6e-4936-ad12-e6499db3eb61\") " pod="cert-manager/cert-manager-webhook-6486ffd997-xmqdc"
Jul 16 11:08:02.860878 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:08:02.860646    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4fe909e4-fe6e-4936-ad12-e6499db3eb61-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-xmqdc\" (UID: \"4fe909e4-fe6e-4936-ad12-e6499db3eb61\") " pod="cert-manager/cert-manager-webhook-6486ffd997-xmqdc"
Jul 16 11:08:02.878776 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:08:02.878743    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4fe909e4-fe6e-4936-ad12-e6499db3eb61-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-xmqdc\" (UID: \"4fe909e4-fe6e-4936-ad12-e6499db3eb61\") " pod="cert-manager/cert-manager-webhook-6486ffd997-xmqdc"
Jul 16 11:08:02.879066 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:08:02.879039    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ljswn\" (UniqueName: \"kubernetes.io/projected/4fe909e4-fe6e-4936-ad12-e6499db3eb61-kube-api-access-ljswn\") pod \"cert-manager-webhook-6486ffd997-xmqdc\" (UID: \"4fe909e4-fe6e-4936-ad12-e6499db3eb61\") " pod="cert-manager/cert-manager-webhook-6486ffd997-xmqdc"
Jul 16 11:08:03.014149 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:08:03.014049    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-xmqdc"
Jul 16 11:08:03.161404 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:08:03.161370    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-xmqdc"]
Jul 16 11:08:03.164430 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:08:03.164391    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4fe909e4_fe6e_4936_ad12_e6499db3eb61.slice/crio-4cf821cd5c140cc334d247e5a77f6c5c3b467ff81d6cb393c7570ac7b249ce4d WatchSource:0}: Error finding container 4cf821cd5c140cc334d247e5a77f6c5c3b467ff81d6cb393c7570ac7b249ce4d: Status 404 returned error can't find the container with id 4cf821cd5c140cc334d247e5a77f6c5c3b467ff81d6cb393c7570ac7b249ce4d
Jul 16 11:08:03.682277 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:08:03.682241    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-xmqdc" event={"ID":"4fe909e4-fe6e-4936-ad12-e6499db3eb61","Type":"ContainerStarted","Data":"4cf821cd5c140cc334d247e5a77f6c5c3b467ff81d6cb393c7570ac7b249ce4d"}
Jul 16 11:08:05.576145 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:08:05.576110    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-jbsp5"]
Jul 16 11:08:05.579052 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:08:05.579018    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-jbsp5"
Jul 16 11:08:05.581667 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:08:05.581641    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-2mcsp\""
Jul 16 11:08:05.590320 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:08:05.590290    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-jbsp5"]
Jul 16 11:08:05.687000 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:08:05.686967    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8nr8b\" (UniqueName: \"kubernetes.io/projected/91dd4aa4-1424-4c95-bdf7-39a2f7d8f0b2-kube-api-access-8nr8b\") pod \"cert-manager-cainjector-559cdd8c87-jbsp5\" (UID: \"91dd4aa4-1424-4c95-bdf7-39a2f7d8f0b2\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-jbsp5"
Jul 16 11:08:05.687181 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:08:05.687015    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/91dd4aa4-1424-4c95-bdf7-39a2f7d8f0b2-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-jbsp5\" (UID: \"91dd4aa4-1424-4c95-bdf7-39a2f7d8f0b2\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-jbsp5"
Jul 16 11:08:05.787982 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:08:05.787872    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8nr8b\" (UniqueName: \"kubernetes.io/projected/91dd4aa4-1424-4c95-bdf7-39a2f7d8f0b2-kube-api-access-8nr8b\") pod \"cert-manager-cainjector-559cdd8c87-jbsp5\" (UID: \"91dd4aa4-1424-4c95-bdf7-39a2f7d8f0b2\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-jbsp5"
Jul 16 11:08:05.787982 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:08:05.787946    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/91dd4aa4-1424-4c95-bdf7-39a2f7d8f0b2-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-jbsp5\" (UID: \"91dd4aa4-1424-4c95-bdf7-39a2f7d8f0b2\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-jbsp5"
Jul 16 11:08:05.802647 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:08:05.802612    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8nr8b\" (UniqueName: \"kubernetes.io/projected/91dd4aa4-1424-4c95-bdf7-39a2f7d8f0b2-kube-api-access-8nr8b\") pod \"cert-manager-cainjector-559cdd8c87-jbsp5\" (UID: \"91dd4aa4-1424-4c95-bdf7-39a2f7d8f0b2\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-jbsp5"
Jul 16 11:08:05.803409 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:08:05.803386    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/91dd4aa4-1424-4c95-bdf7-39a2f7d8f0b2-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-jbsp5\" (UID: \"91dd4aa4-1424-4c95-bdf7-39a2f7d8f0b2\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-jbsp5"
Jul 16 11:08:05.891544 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:08:05.891397    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-jbsp5"
Jul 16 11:08:06.691161 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:08:06.691136    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-jbsp5"]
Jul 16 11:08:06.694399 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:08:06.694369    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod91dd4aa4_1424_4c95_bdf7_39a2f7d8f0b2.slice/crio-c971531528fe5275e96525d3456c113b932ced722a30c8980990f78b0109a34a WatchSource:0}: Error finding container c971531528fe5275e96525d3456c113b932ced722a30c8980990f78b0109a34a: Status 404 returned error can't find the container with id c971531528fe5275e96525d3456c113b932ced722a30c8980990f78b0109a34a
Jul 16 11:08:06.697123 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:08:06.697094    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-xmqdc" event={"ID":"4fe909e4-fe6e-4936-ad12-e6499db3eb61","Type":"ContainerStarted","Data":"3cdf789161b3fc8ede42fa552608a9d0aa3553203c1d3bf0812f0a0990612c4c"}
Jul 16 11:08:06.697244 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:08:06.697226    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-xmqdc"
Jul 16 11:08:06.714097 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:08:06.714049    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-xmqdc" podStartSLOduration=1.286251811 podStartE2EDuration="4.714034101s" totalImagesPullingTime="3.42778229s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 11:08:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 11:08:06.71294252 +0000 UTC m=+279.347164003" watchObservedRunningTime="2026-07-16 11:08:06.714034101 +0000 UTC m=+279.348255555"
Jul 16 11:08:07.702869 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:08:07.702831    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-jbsp5" event={"ID":"91dd4aa4-1424-4c95-bdf7-39a2f7d8f0b2","Type":"ContainerStarted","Data":"681b4de91a2771142ad3df080d231c36fa069e8d16534032b3d247ad6ef9cc8b"}
Jul 16 11:08:07.703345 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:08:07.702884    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-jbsp5" event={"ID":"91dd4aa4-1424-4c95-bdf7-39a2f7d8f0b2","Type":"ContainerStarted","Data":"c971531528fe5275e96525d3456c113b932ced722a30c8980990f78b0109a34a"}
Jul 16 11:08:07.723710 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:08:07.723658    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-jbsp5" podStartSLOduration=2.7236424 podStartE2EDuration="2.7236424s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 11:08:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 11:08:07.722015811 +0000 UTC m=+280.356237266" watchObservedRunningTime="2026-07-16 11:08:07.7236424 +0000 UTC m=+280.357863852"
Jul 16 11:08:12.705840 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:08:12.705805    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-xmqdc"
Jul 16 11:08:29.096072 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:08:29.096043    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-9lm5s_f64f15a3-5a9f-4737-bc4f-73d5ec311e41/console-operator/1.log"
Jul 16 11:08:29.096504 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:08:29.096080    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-9lm5s_f64f15a3-5a9f-4737-bc4f-73d5ec311e41/console-operator/1.log"
Jul 16 11:08:29.104124 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:08:29.103968    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2db6t_69aed169-e8af-429d-99c3-3b20be64e351/ovn-acl-logging/0.log"
Jul 16 11:08:29.104124 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:08:29.103980    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2db6t_69aed169-e8af-429d-99c3-3b20be64e351/ovn-acl-logging/0.log"
Jul 16 11:08:29.113054 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:08:29.113028    2583 kubelet.go:1648] "Image garbage collection succeeded"
Jul 16 11:08:40.528332 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:08:40.528290    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-5dbfbcbfc5-564zb"]
Jul 16 11:08:40.532093 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:08:40.532075    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5dbfbcbfc5-564zb"
Jul 16 11:08:40.538613 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:08:40.538576    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 16 11:08:40.538868 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:08:40.538845    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 16 11:08:40.539480 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:08:40.539457    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 16 11:08:40.539480 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:08:40.539462    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 16 11:08:40.539686 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:08:40.539541    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-mdb7c\""
Jul 16 11:08:40.539851 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:08:40.539837    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 16 11:08:40.546590 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:08:40.546555    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5dbfbcbfc5-564zb"]
Jul 16 11:08:40.561290 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:08:40.561255    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/c85bab5b-1491-4e0c-b07a-742c22bb7a9e-manager-config\") pod \"lws-controller-manager-5dbfbcbfc5-564zb\" (UID: \"c85bab5b-1491-4e0c-b07a-742c22bb7a9e\") " pod="openshift-lws-operator/lws-controller-manager-5dbfbcbfc5-564zb"
Jul 16 11:08:40.561290 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:08:40.561289    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c85bab5b-1491-4e0c-b07a-742c22bb7a9e-cert\") pod \"lws-controller-manager-5dbfbcbfc5-564zb\" (UID: \"c85bab5b-1491-4e0c-b07a-742c22bb7a9e\") " pod="openshift-lws-operator/lws-controller-manager-5dbfbcbfc5-564zb"
Jul 16 11:08:40.561469 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:08:40.561308    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qj52s\" (UniqueName: \"kubernetes.io/projected/c85bab5b-1491-4e0c-b07a-742c22bb7a9e-kube-api-access-qj52s\") pod \"lws-controller-manager-5dbfbcbfc5-564zb\" (UID: \"c85bab5b-1491-4e0c-b07a-742c22bb7a9e\") " pod="openshift-lws-operator/lws-controller-manager-5dbfbcbfc5-564zb"
Jul 16 11:08:40.561469 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:08:40.561435    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/c85bab5b-1491-4e0c-b07a-742c22bb7a9e-metrics-cert\") pod \"lws-controller-manager-5dbfbcbfc5-564zb\" (UID: \"c85bab5b-1491-4e0c-b07a-742c22bb7a9e\") " pod="openshift-lws-operator/lws-controller-manager-5dbfbcbfc5-564zb"
Jul 16 11:08:40.661800 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:08:40.661765    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/c85bab5b-1491-4e0c-b07a-742c22bb7a9e-manager-config\") pod \"lws-controller-manager-5dbfbcbfc5-564zb\" (UID: \"c85bab5b-1491-4e0c-b07a-742c22bb7a9e\") " pod="openshift-lws-operator/lws-controller-manager-5dbfbcbfc5-564zb"
Jul 16 11:08:40.661800 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:08:40.661799    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c85bab5b-1491-4e0c-b07a-742c22bb7a9e-cert\") pod \"lws-controller-manager-5dbfbcbfc5-564zb\" (UID: \"c85bab5b-1491-4e0c-b07a-742c22bb7a9e\") " pod="openshift-lws-operator/lws-controller-manager-5dbfbcbfc5-564zb"
Jul 16 11:08:40.662013 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:08:40.661816    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qj52s\" (UniqueName: \"kubernetes.io/projected/c85bab5b-1491-4e0c-b07a-742c22bb7a9e-kube-api-access-qj52s\") pod \"lws-controller-manager-5dbfbcbfc5-564zb\" (UID: \"c85bab5b-1491-4e0c-b07a-742c22bb7a9e\") " pod="openshift-lws-operator/lws-controller-manager-5dbfbcbfc5-564zb"
Jul 16 11:08:40.662013 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:08:40.661872    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/c85bab5b-1491-4e0c-b07a-742c22bb7a9e-metrics-cert\") pod \"lws-controller-manager-5dbfbcbfc5-564zb\" (UID: \"c85bab5b-1491-4e0c-b07a-742c22bb7a9e\") " pod="openshift-lws-operator/lws-controller-manager-5dbfbcbfc5-564zb"
Jul 16 11:08:40.662388 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:08:40.662360    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/c85bab5b-1491-4e0c-b07a-742c22bb7a9e-manager-config\") pod \"lws-controller-manager-5dbfbcbfc5-564zb\" (UID: \"c85bab5b-1491-4e0c-b07a-742c22bb7a9e\") " pod="openshift-lws-operator/lws-controller-manager-5dbfbcbfc5-564zb"
Jul 16 11:08:40.664267 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:08:40.664235    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/c85bab5b-1491-4e0c-b07a-742c22bb7a9e-metrics-cert\") pod \"lws-controller-manager-5dbfbcbfc5-564zb\" (UID: \"c85bab5b-1491-4e0c-b07a-742c22bb7a9e\") " pod="openshift-lws-operator/lws-controller-manager-5dbfbcbfc5-564zb"
Jul 16 11:08:40.664367 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:08:40.664292    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c85bab5b-1491-4e0c-b07a-742c22bb7a9e-cert\") pod \"lws-controller-manager-5dbfbcbfc5-564zb\" (UID: \"c85bab5b-1491-4e0c-b07a-742c22bb7a9e\") " pod="openshift-lws-operator/lws-controller-manager-5dbfbcbfc5-564zb"
Jul 16 11:08:40.684234 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:08:40.684204    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qj52s\" (UniqueName: \"kubernetes.io/projected/c85bab5b-1491-4e0c-b07a-742c22bb7a9e-kube-api-access-qj52s\") pod \"lws-controller-manager-5dbfbcbfc5-564zb\" (UID: \"c85bab5b-1491-4e0c-b07a-742c22bb7a9e\") " pod="openshift-lws-operator/lws-controller-manager-5dbfbcbfc5-564zb"
Jul 16 11:08:40.841236 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:08:40.841129    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5dbfbcbfc5-564zb"
Jul 16 11:08:40.999564 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:08:40.999531    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc85bab5b_1491_4e0c_b07a_742c22bb7a9e.slice/crio-865836e602a3c4c63bb59a9b83777a44c4f49cec15641c0637c7ea79db46e0cd WatchSource:0}: Error finding container 865836e602a3c4c63bb59a9b83777a44c4f49cec15641c0637c7ea79db46e0cd: Status 404 returned error can't find the container with id 865836e602a3c4c63bb59a9b83777a44c4f49cec15641c0637c7ea79db46e0cd
Jul 16 11:08:41.001792 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:08:41.001768    2583 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 16 11:08:41.002121 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:08:41.002098    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5dbfbcbfc5-564zb"]
Jul 16 11:08:41.856659 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:08:41.856560    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5dbfbcbfc5-564zb" event={"ID":"c85bab5b-1491-4e0c-b07a-742c22bb7a9e","Type":"ContainerStarted","Data":"865836e602a3c4c63bb59a9b83777a44c4f49cec15641c0637c7ea79db46e0cd"}
Jul 16 11:08:42.673552 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:08:42.673508    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-58ffff6c5d-xgzpm"]
Jul 16 11:08:42.683499 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:08:42.683467    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-58ffff6c5d-xgzpm"
Jul 16 11:08:42.688525 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:08:42.688496    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 16 11:08:42.688903 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:08:42.688867    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-zqz4c\""
Jul 16 11:08:42.689235 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:08:42.689214    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 16 11:08:42.689340 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:08:42.689236    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 16 11:08:42.689340 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:08:42.689283    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 16 11:08:42.693968 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:08:42.693940    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-58ffff6c5d-xgzpm"]
Jul 16 11:08:42.780770 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:08:42.780728    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/a71243ff-abce-4248-8942-fc01f1e7ffab-manifests\") pod \"opendatahub-operator-controller-manager-58ffff6c5d-xgzpm\" (UID: \"a71243ff-abce-4248-8942-fc01f1e7ffab\") " pod="opendatahub/opendatahub-operator-controller-manager-58ffff6c5d-xgzpm"
Jul 16 11:08:42.780770 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:08:42.780786    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a71243ff-abce-4248-8942-fc01f1e7ffab-webhook-cert\") pod \"opendatahub-operator-controller-manager-58ffff6c5d-xgzpm\" (UID: \"a71243ff-abce-4248-8942-fc01f1e7ffab\") " pod="opendatahub/opendatahub-operator-controller-manager-58ffff6c5d-xgzpm"
Jul 16 11:08:42.781035 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:08:42.780897    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a71243ff-abce-4248-8942-fc01f1e7ffab-tmp\") pod \"opendatahub-operator-controller-manager-58ffff6c5d-xgzpm\" (UID: \"a71243ff-abce-4248-8942-fc01f1e7ffab\") " pod="opendatahub/opendatahub-operator-controller-manager-58ffff6c5d-xgzpm"
Jul 16 11:08:42.781035 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:08:42.780941    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n5pst\" (UniqueName: \"kubernetes.io/projected/a71243ff-abce-4248-8942-fc01f1e7ffab-kube-api-access-n5pst\") pod \"opendatahub-operator-controller-manager-58ffff6c5d-xgzpm\" (UID: \"a71243ff-abce-4248-8942-fc01f1e7ffab\") " pod="opendatahub/opendatahub-operator-controller-manager-58ffff6c5d-xgzpm"
Jul 16 11:08:42.781035 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:08:42.781005    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/a71243ff-abce-4248-8942-fc01f1e7ffab-apiservice-cert\") pod \"opendatahub-operator-controller-manager-58ffff6c5d-xgzpm\" (UID: \"a71243ff-abce-4248-8942-fc01f1e7ffab\") " pod="opendatahub/opendatahub-operator-controller-manager-58ffff6c5d-xgzpm"
Jul 16 11:08:42.882021 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:08:42.881980    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a71243ff-abce-4248-8942-fc01f1e7ffab-tmp\") pod \"opendatahub-operator-controller-manager-58ffff6c5d-xgzpm\" (UID: \"a71243ff-abce-4248-8942-fc01f1e7ffab\") " pod="opendatahub/opendatahub-operator-controller-manager-58ffff6c5d-xgzpm"
Jul 16 11:08:42.882485 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:08:42.882054    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n5pst\" (UniqueName: \"kubernetes.io/projected/a71243ff-abce-4248-8942-fc01f1e7ffab-kube-api-access-n5pst\") pod \"opendatahub-operator-controller-manager-58ffff6c5d-xgzpm\" (UID: \"a71243ff-abce-4248-8942-fc01f1e7ffab\") " pod="opendatahub/opendatahub-operator-controller-manager-58ffff6c5d-xgzpm"
Jul 16 11:08:42.882485 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:08:42.882087    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/a71243ff-abce-4248-8942-fc01f1e7ffab-apiservice-cert\") pod \"opendatahub-operator-controller-manager-58ffff6c5d-xgzpm\" (UID: \"a71243ff-abce-4248-8942-fc01f1e7ffab\") " pod="opendatahub/opendatahub-operator-controller-manager-58ffff6c5d-xgzpm"
Jul 16 11:08:42.882485 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:08:42.882125    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/a71243ff-abce-4248-8942-fc01f1e7ffab-manifests\") pod \"opendatahub-operator-controller-manager-58ffff6c5d-xgzpm\" (UID: \"a71243ff-abce-4248-8942-fc01f1e7ffab\") " pod="opendatahub/opendatahub-operator-controller-manager-58ffff6c5d-xgzpm"
Jul 16 11:08:42.882485 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:08:42.882146    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a71243ff-abce-4248-8942-fc01f1e7ffab-webhook-cert\") pod \"opendatahub-operator-controller-manager-58ffff6c5d-xgzpm\" (UID: \"a71243ff-abce-4248-8942-fc01f1e7ffab\") " pod="opendatahub/opendatahub-operator-controller-manager-58ffff6c5d-xgzpm"
Jul 16 11:08:42.882747 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:08:42.882663    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/a71243ff-abce-4248-8942-fc01f1e7ffab-manifests\") pod \"opendatahub-operator-controller-manager-58ffff6c5d-xgzpm\" (UID: \"a71243ff-abce-4248-8942-fc01f1e7ffab\") " pod="opendatahub/opendatahub-operator-controller-manager-58ffff6c5d-xgzpm"
Jul 16 11:08:42.884509 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:08:42.884475    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a71243ff-abce-4248-8942-fc01f1e7ffab-tmp\") pod \"opendatahub-operator-controller-manager-58ffff6c5d-xgzpm\" (UID: \"a71243ff-abce-4248-8942-fc01f1e7ffab\") " pod="opendatahub/opendatahub-operator-controller-manager-58ffff6c5d-xgzpm"
Jul 16 11:08:42.884700 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:08:42.884681    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/a71243ff-abce-4248-8942-fc01f1e7ffab-apiservice-cert\") pod \"opendatahub-operator-controller-manager-58ffff6c5d-xgzpm\" (UID: \"a71243ff-abce-4248-8942-fc01f1e7ffab\") " pod="opendatahub/opendatahub-operator-controller-manager-58ffff6c5d-xgzpm"
Jul 16 11:08:42.884836 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:08:42.884807    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a71243ff-abce-4248-8942-fc01f1e7ffab-webhook-cert\") pod \"opendatahub-operator-controller-manager-58ffff6c5d-xgzpm\" (UID: \"a71243ff-abce-4248-8942-fc01f1e7ffab\") " pod="opendatahub/opendatahub-operator-controller-manager-58ffff6c5d-xgzpm"
Jul 16 11:08:42.899715 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:08:42.899677    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n5pst\" (UniqueName: \"kubernetes.io/projected/a71243ff-abce-4248-8942-fc01f1e7ffab-kube-api-access-n5pst\") pod \"opendatahub-operator-controller-manager-58ffff6c5d-xgzpm\" (UID: \"a71243ff-abce-4248-8942-fc01f1e7ffab\") " pod="opendatahub/opendatahub-operator-controller-manager-58ffff6c5d-xgzpm"
Jul 16 11:08:42.999142 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:08:42.999053    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-58ffff6c5d-xgzpm"
Jul 16 11:08:43.405191 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:08:43.405168    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-58ffff6c5d-xgzpm"]
Jul 16 11:08:43.408431 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:08:43.408407    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda71243ff_abce_4248_8942_fc01f1e7ffab.slice/crio-80b49e7980b6b064da572ba1e1e5104467a74c6a03c2acb0b8a55f3c1c2f7aa3 WatchSource:0}: Error finding container 80b49e7980b6b064da572ba1e1e5104467a74c6a03c2acb0b8a55f3c1c2f7aa3: Status 404 returned error can't find the container with id 80b49e7980b6b064da572ba1e1e5104467a74c6a03c2acb0b8a55f3c1c2f7aa3
Jul 16 11:08:43.870972 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:08:43.870757    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5dbfbcbfc5-564zb" event={"ID":"c85bab5b-1491-4e0c-b07a-742c22bb7a9e","Type":"ContainerStarted","Data":"91eedcafdb4832f5fd65e221b9a32c2598965be2834faacb4c803e95758cf6bf"}
Jul 16 11:08:43.870972 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:08:43.870946    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-5dbfbcbfc5-564zb"
Jul 16 11:08:43.873139 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:08:43.873071    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-58ffff6c5d-xgzpm" event={"ID":"a71243ff-abce-4248-8942-fc01f1e7ffab","Type":"ContainerStarted","Data":"80b49e7980b6b064da572ba1e1e5104467a74c6a03c2acb0b8a55f3c1c2f7aa3"}
Jul 16 11:08:43.913349 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:08:43.913280    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-5dbfbcbfc5-564zb" podStartSLOduration=1.58554141 podStartE2EDuration="3.913255162s" totalImagesPullingTime="2.327713752s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 11:08:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 11:08:43.910769355 +0000 UTC m=+316.544990836" watchObservedRunningTime="2026-07-16 11:08:43.913255162 +0000 UTC m=+316.547476619"
Jul 16 11:08:46.891461 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:08:46.891419    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-58ffff6c5d-xgzpm" event={"ID":"a71243ff-abce-4248-8942-fc01f1e7ffab","Type":"ContainerStarted","Data":"f38fc33ddebbe3031c7e9cbdfba9963218a90e00cb52e2c330d87687231d984e"}
Jul 16 11:08:47.896984 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:08:47.896944    2583 generic.go:356] "Generic (PLEG): container finished" podID="a71243ff-abce-4248-8942-fc01f1e7ffab" containerID="f38fc33ddebbe3031c7e9cbdfba9963218a90e00cb52e2c330d87687231d984e" exitCode=0
Jul 16 11:08:47.897385 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:08:47.897028    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-58ffff6c5d-xgzpm" event={"ID":"a71243ff-abce-4248-8942-fc01f1e7ffab","Type":"ContainerDied","Data":"f38fc33ddebbe3031c7e9cbdfba9963218a90e00cb52e2c330d87687231d984e"}
Jul 16 11:08:48.903811 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:08:48.903773    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-58ffff6c5d-xgzpm" event={"ID":"a71243ff-abce-4248-8942-fc01f1e7ffab","Type":"ContainerStarted","Data":"38c0fbdfa152b2519c8ef3733d40f47016707b5a6b54bb7088518cda79aa1d12"}
Jul 16 11:08:48.904201 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:08:48.903875    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-58ffff6c5d-xgzpm"
Jul 16 11:08:48.935508 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:08:48.935455    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-58ffff6c5d-xgzpm" podStartSLOduration=2.782624397 podStartE2EDuration="6.935434142s" totalImagesPullingTime="3.056102644s" totalInitContainerRuntime="1.096707101s" isStatefulPod=false podCreationTimestamp="2026-07-16 11:08:42 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 11:08:48.933900577 +0000 UTC m=+321.568122021" watchObservedRunningTime="2026-07-16 11:08:48.935434142 +0000 UTC m=+321.569655596"
Jul 16 11:08:54.882708 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:08:54.882678    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-5dbfbcbfc5-564zb"
Jul 16 11:08:59.911395 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:08:59.911361    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-58ffff6c5d-xgzpm"
Jul 16 11:09:10.439706 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:09:10.439668    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4cfrcl"]
Jul 16 11:09:10.447400 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:09:10.447368    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4cfrcl"
Jul 16 11:09:10.450175 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:09:10.450151    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 16 11:09:10.450317 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:09:10.450157    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-gf6w7\""
Jul 16 11:09:10.458315 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:09:10.458288    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4cfrcl"]
Jul 16 11:09:10.517585 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:09:10.517545    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/44e90033-3f7d-4432-941d-646531a6f9f4-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4cfrcl\" (UID: \"44e90033-3f7d-4432-941d-646531a6f9f4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4cfrcl"
Jul 16 11:09:10.517585 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:09:10.517585    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/44e90033-3f7d-4432-941d-646531a6f9f4-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4cfrcl\" (UID: \"44e90033-3f7d-4432-941d-646531a6f9f4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4cfrcl"
Jul 16 11:09:10.517835 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:09:10.517673    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/44e90033-3f7d-4432-941d-646531a6f9f4-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4cfrcl\" (UID: \"44e90033-3f7d-4432-941d-646531a6f9f4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4cfrcl"
Jul 16 11:09:10.517835 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:09:10.517696    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/44e90033-3f7d-4432-941d-646531a6f9f4-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4cfrcl\" (UID: \"44e90033-3f7d-4432-941d-646531a6f9f4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4cfrcl"
Jul 16 11:09:10.517835 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:09:10.517713    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/44e90033-3f7d-4432-941d-646531a6f9f4-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4cfrcl\" (UID: \"44e90033-3f7d-4432-941d-646531a6f9f4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4cfrcl"
Jul 16 11:09:10.517835 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:09:10.517736    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/44e90033-3f7d-4432-941d-646531a6f9f4-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4cfrcl\" (UID: \"44e90033-3f7d-4432-941d-646531a6f9f4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4cfrcl"
Jul 16 11:09:10.517835 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:09:10.517757    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/44e90033-3f7d-4432-941d-646531a6f9f4-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4cfrcl\" (UID: \"44e90033-3f7d-4432-941d-646531a6f9f4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4cfrcl"
Jul 16 11:09:10.517835 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:09:10.517774    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n5czg\" (UniqueName: \"kubernetes.io/projected/44e90033-3f7d-4432-941d-646531a6f9f4-kube-api-access-n5czg\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4cfrcl\" (UID: \"44e90033-3f7d-4432-941d-646531a6f9f4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4cfrcl"
Jul 16 11:09:10.518082 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:09:10.517841    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/44e90033-3f7d-4432-941d-646531a6f9f4-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4cfrcl\" (UID: \"44e90033-3f7d-4432-941d-646531a6f9f4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4cfrcl"
Jul 16 11:09:10.618424 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:09:10.618389    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/44e90033-3f7d-4432-941d-646531a6f9f4-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4cfrcl\" (UID: \"44e90033-3f7d-4432-941d-646531a6f9f4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4cfrcl"
Jul 16 11:09:10.618424 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:09:10.618424    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/44e90033-3f7d-4432-941d-646531a6f9f4-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4cfrcl\" (UID: \"44e90033-3f7d-4432-941d-646531a6f9f4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4cfrcl"
Jul 16 11:09:10.618699 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:09:10.618649    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/44e90033-3f7d-4432-941d-646531a6f9f4-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4cfrcl\" (UID: \"44e90033-3f7d-4432-941d-646531a6f9f4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4cfrcl"
Jul 16 11:09:10.618752 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:09:10.618707    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/44e90033-3f7d-4432-941d-646531a6f9f4-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4cfrcl\" (UID: \"44e90033-3f7d-4432-941d-646531a6f9f4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4cfrcl"
Jul 16 11:09:10.618752 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:09:10.618740    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/44e90033-3f7d-4432-941d-646531a6f9f4-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4cfrcl\" (UID: \"44e90033-3f7d-4432-941d-646531a6f9f4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4cfrcl"
Jul 16 11:09:10.618840 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:09:10.618777    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/44e90033-3f7d-4432-941d-646531a6f9f4-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4cfrcl\" (UID: \"44e90033-3f7d-4432-941d-646531a6f9f4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4cfrcl"
Jul 16 11:09:10.618893 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:09:10.618869    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/44e90033-3f7d-4432-941d-646531a6f9f4-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4cfrcl\" (UID: \"44e90033-3f7d-4432-941d-646531a6f9f4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4cfrcl"
Jul 16 11:09:10.618951 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:09:10.618914    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/44e90033-3f7d-4432-941d-646531a6f9f4-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4cfrcl\" (UID: \"44e90033-3f7d-4432-941d-646531a6f9f4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4cfrcl"
Jul 16 11:09:10.619013 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:09:10.618952    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n5czg\" (UniqueName: \"kubernetes.io/projected/44e90033-3f7d-4432-941d-646531a6f9f4-kube-api-access-n5czg\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4cfrcl\" (UID: \"44e90033-3f7d-4432-941d-646531a6f9f4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4cfrcl"
Jul 16 11:09:10.619013 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:09:10.618971    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/44e90033-3f7d-4432-941d-646531a6f9f4-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4cfrcl\" (UID: \"44e90033-3f7d-4432-941d-646531a6f9f4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4cfrcl"
Jul 16 11:09:10.619117 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:09:10.619027    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/44e90033-3f7d-4432-941d-646531a6f9f4-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4cfrcl\" (UID: \"44e90033-3f7d-4432-941d-646531a6f9f4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4cfrcl"
Jul 16 11:09:10.619307 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:09:10.619272    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/44e90033-3f7d-4432-941d-646531a6f9f4-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4cfrcl\" (UID: \"44e90033-3f7d-4432-941d-646531a6f9f4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4cfrcl"
Jul 16 11:09:10.619468 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:09:10.619274    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/44e90033-3f7d-4432-941d-646531a6f9f4-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4cfrcl\" (UID: \"44e90033-3f7d-4432-941d-646531a6f9f4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4cfrcl"
Jul 16 11:09:10.619686 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:09:10.619666    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/44e90033-3f7d-4432-941d-646531a6f9f4-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4cfrcl\" (UID: \"44e90033-3f7d-4432-941d-646531a6f9f4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4cfrcl"
Jul 16 11:09:10.621011 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:09:10.620989    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/44e90033-3f7d-4432-941d-646531a6f9f4-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4cfrcl\" (UID: \"44e90033-3f7d-4432-941d-646531a6f9f4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4cfrcl"
Jul 16 11:09:10.621396 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:09:10.621374    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/44e90033-3f7d-4432-941d-646531a6f9f4-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4cfrcl\" (UID: \"44e90033-3f7d-4432-941d-646531a6f9f4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4cfrcl"
Jul 16 11:09:10.641368 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:09:10.641331    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n5czg\" (UniqueName: \"kubernetes.io/projected/44e90033-3f7d-4432-941d-646531a6f9f4-kube-api-access-n5czg\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4cfrcl\" (UID: \"44e90033-3f7d-4432-941d-646531a6f9f4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4cfrcl"
Jul 16 11:09:10.644546 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:09:10.644514    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/44e90033-3f7d-4432-941d-646531a6f9f4-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4cfrcl\" (UID: \"44e90033-3f7d-4432-941d-646531a6f9f4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4cfrcl"
Jul 16 11:09:10.759662 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:09:10.759538    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4cfrcl"
Jul 16 11:09:10.927158 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:09:10.927081    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4cfrcl"]
Jul 16 11:09:10.931108 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:09:10.931076    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44e90033_3f7d_4432_941d_646531a6f9f4.slice/crio-0cc24670fd060a2078ebdcb6d276946ec79fd32a81ca930adbf2de78783b1126 WatchSource:0}: Error finding container 0cc24670fd060a2078ebdcb6d276946ec79fd32a81ca930adbf2de78783b1126: Status 404 returned error can't find the container with id 0cc24670fd060a2078ebdcb6d276946ec79fd32a81ca930adbf2de78783b1126
Jul 16 11:09:11.009040 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:09:11.009003    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4cfrcl" event={"ID":"44e90033-3f7d-4432-941d-646531a6f9f4","Type":"ContainerStarted","Data":"0cc24670fd060a2078ebdcb6d276946ec79fd32a81ca930adbf2de78783b1126"}
Jul 16 11:09:11.759272 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:09:11.759232    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-txnpn"]
Jul 16 11:09:11.762901 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:09:11.762874    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-txnpn"
Jul 16 11:09:11.765582 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:09:11.765404    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 16 11:09:11.765582 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:09:11.765452    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 16 11:09:11.765582 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:09:11.765534    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-8gmn9\""
Jul 16 11:09:11.765871 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:09:11.765793    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 16 11:09:11.775224 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:09:11.775141    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-txnpn"]
Jul 16 11:09:11.831971 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:09:11.831932    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/af6965b3-3b33-4a0d-bbf8-d18c7e41c21b-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-txnpn\" (UID: \"af6965b3-3b33-4a0d-bbf8-d18c7e41c21b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-txnpn"
Jul 16 11:09:11.831971 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:09:11.831974    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/af6965b3-3b33-4a0d-bbf8-d18c7e41c21b-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-txnpn\" (UID: \"af6965b3-3b33-4a0d-bbf8-d18c7e41c21b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-txnpn"
Jul 16 11:09:11.832228 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:09:11.831998    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5dq9z\" (UniqueName: \"kubernetes.io/projected/af6965b3-3b33-4a0d-bbf8-d18c7e41c21b-kube-api-access-5dq9z\") pod \"istiod-openshift-gateway-94bb8fbfd-txnpn\" (UID: \"af6965b3-3b33-4a0d-bbf8-d18c7e41c21b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-txnpn"
Jul 16 11:09:11.832228 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:09:11.832107    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/af6965b3-3b33-4a0d-bbf8-d18c7e41c21b-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-txnpn\" (UID: \"af6965b3-3b33-4a0d-bbf8-d18c7e41c21b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-txnpn"
Jul 16 11:09:11.832228 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:09:11.832182    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/af6965b3-3b33-4a0d-bbf8-d18c7e41c21b-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-txnpn\" (UID: \"af6965b3-3b33-4a0d-bbf8-d18c7e41c21b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-txnpn"
Jul 16 11:09:11.832228 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:09:11.832214    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/af6965b3-3b33-4a0d-bbf8-d18c7e41c21b-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-txnpn\" (UID: \"af6965b3-3b33-4a0d-bbf8-d18c7e41c21b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-txnpn"
Jul 16 11:09:11.832473 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:09:11.832241    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/af6965b3-3b33-4a0d-bbf8-d18c7e41c21b-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-txnpn\" (UID: \"af6965b3-3b33-4a0d-bbf8-d18c7e41c21b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-txnpn"
Jul 16 11:09:11.934082 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:09:11.933365    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5dq9z\" (UniqueName: \"kubernetes.io/projected/af6965b3-3b33-4a0d-bbf8-d18c7e41c21b-kube-api-access-5dq9z\") pod \"istiod-openshift-gateway-94bb8fbfd-txnpn\" (UID: \"af6965b3-3b33-4a0d-bbf8-d18c7e41c21b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-txnpn"
Jul 16 11:09:11.934082 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:09:11.933442    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/af6965b3-3b33-4a0d-bbf8-d18c7e41c21b-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-txnpn\" (UID: \"af6965b3-3b33-4a0d-bbf8-d18c7e41c21b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-txnpn"
Jul 16 11:09:11.934082 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:09:11.933503    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/af6965b3-3b33-4a0d-bbf8-d18c7e41c21b-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-txnpn\" (UID: \"af6965b3-3b33-4a0d-bbf8-d18c7e41c21b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-txnpn"
Jul 16 11:09:11.934082 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:09:11.933532    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/af6965b3-3b33-4a0d-bbf8-d18c7e41c21b-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-txnpn\" (UID: \"af6965b3-3b33-4a0d-bbf8-d18c7e41c21b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-txnpn"
Jul 16 11:09:11.934082 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:09:11.933553    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/af6965b3-3b33-4a0d-bbf8-d18c7e41c21b-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-txnpn\" (UID: \"af6965b3-3b33-4a0d-bbf8-d18c7e41c21b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-txnpn"
Jul 16 11:09:11.934082 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:09:11.933627    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/af6965b3-3b33-4a0d-bbf8-d18c7e41c21b-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-txnpn\" (UID: \"af6965b3-3b33-4a0d-bbf8-d18c7e41c21b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-txnpn"
Jul 16 11:09:11.934082 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:09:11.933651    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/af6965b3-3b33-4a0d-bbf8-d18c7e41c21b-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-txnpn\" (UID: \"af6965b3-3b33-4a0d-bbf8-d18c7e41c21b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-txnpn"
Jul 16 11:09:11.934559 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:09:11.934374    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/af6965b3-3b33-4a0d-bbf8-d18c7e41c21b-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-txnpn\" (UID: \"af6965b3-3b33-4a0d-bbf8-d18c7e41c21b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-txnpn"
Jul 16 11:09:11.937212 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:09:11.937185    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/af6965b3-3b33-4a0d-bbf8-d18c7e41c21b-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-txnpn\" (UID: \"af6965b3-3b33-4a0d-bbf8-d18c7e41c21b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-txnpn"
Jul 16 11:09:11.937391 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:09:11.937367    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/af6965b3-3b33-4a0d-bbf8-d18c7e41c21b-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-txnpn\" (UID: \"af6965b3-3b33-4a0d-bbf8-d18c7e41c21b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-txnpn"
Jul 16 11:09:11.937453 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:09:11.937406    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/af6965b3-3b33-4a0d-bbf8-d18c7e41c21b-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-txnpn\" (UID: \"af6965b3-3b33-4a0d-bbf8-d18c7e41c21b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-txnpn"
Jul 16 11:09:11.938425 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:09:11.938402    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/af6965b3-3b33-4a0d-bbf8-d18c7e41c21b-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-txnpn\" (UID: \"af6965b3-3b33-4a0d-bbf8-d18c7e41c21b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-txnpn"
Jul 16 11:09:11.952039 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:09:11.951989    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5dq9z\" (UniqueName: \"kubernetes.io/projected/af6965b3-3b33-4a0d-bbf8-d18c7e41c21b-kube-api-access-5dq9z\") pod \"istiod-openshift-gateway-94bb8fbfd-txnpn\" (UID: \"af6965b3-3b33-4a0d-bbf8-d18c7e41c21b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-txnpn"
Jul 16 11:09:11.952222 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:09:11.952198    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/af6965b3-3b33-4a0d-bbf8-d18c7e41c21b-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-txnpn\" (UID: \"af6965b3-3b33-4a0d-bbf8-d18c7e41c21b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-txnpn"
Jul 16 11:09:12.078891 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:09:12.078792    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-txnpn"
Jul 16 11:09:12.287047 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:09:12.287017    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-txnpn"]
Jul 16 11:09:12.289336 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:09:12.289307    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf6965b3_3b33_4a0d_bbf8_d18c7e41c21b.slice/crio-4811095fd8fec395afba1d67577bc523551d0c3537de78ac34f853034cbde83a WatchSource:0}: Error finding container 4811095fd8fec395afba1d67577bc523551d0c3537de78ac34f853034cbde83a: Status 404 returned error can't find the container with id 4811095fd8fec395afba1d67577bc523551d0c3537de78ac34f853034cbde83a
Jul 16 11:09:13.023387 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:09:13.023345    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-txnpn" event={"ID":"af6965b3-3b33-4a0d-bbf8-d18c7e41c21b","Type":"ContainerStarted","Data":"4811095fd8fec395afba1d67577bc523551d0c3537de78ac34f853034cbde83a"}
Jul 16 11:09:13.698051 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:09:13.698002    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 11:09:13.698175 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:09:13.698092    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 11:09:13.698175 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:09:13.698122    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 11:09:14.030206 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:09:14.030109    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4cfrcl" event={"ID":"44e90033-3f7d-4432-941d-646531a6f9f4","Type":"ContainerStarted","Data":"20ba2b56300190bf8a8ffb1fa9a6ac787a364b19870f308476082ad05fe81ca2"}
Jul 16 11:09:14.062954 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:09:14.062901    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4cfrcl" podStartSLOduration=1.298509026 podStartE2EDuration="4.062884854s" totalImagesPullingTime="2.764375828s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 11:09:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 11:09:14.059518165 +0000 UTC m=+346.693739619" watchObservedRunningTime="2026-07-16 11:09:14.062884854 +0000 UTC m=+346.697106307"
Jul 16 11:09:14.761006 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:09:14.760910    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4cfrcl"
Jul 16 11:09:14.766531 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:09:14.766500    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4cfrcl"
Jul 16 11:09:15.037420 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:09:15.036948    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4cfrcl"
Jul 16 11:09:15.038672 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:09:15.038649    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4cfrcl"
Jul 16 11:09:16.525560 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:09:16.525526    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 11:09:16.525898 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:09:16.525589    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 11:09:17.050516 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:09:17.050464    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-txnpn" event={"ID":"af6965b3-3b33-4a0d-bbf8-d18c7e41c21b","Type":"ContainerStarted","Data":"b343bc734496aebf32fafb2e532f07e29936b3c7584fad68d7418e1ca11f4b60"}
Jul 16 11:09:17.051070 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:09:17.051025    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-txnpn"
Jul 16 11:09:17.052871 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:09:17.052817    2583 patch_prober.go:28] interesting pod/istiod-openshift-gateway-94bb8fbfd-txnpn container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 16 11:09:17.052985 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:09:17.052887    2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-txnpn" podUID="af6965b3-3b33-4a0d-bbf8-d18c7e41c21b" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 16 11:09:18.055163 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:09:18.055133    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-txnpn"
Jul 16 11:09:18.077359 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:09:18.077304    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-txnpn" podStartSLOduration=2.844543859 podStartE2EDuration="7.077286688s" totalImagesPullingTime="4.232742829s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 11:09:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 11:09:17.096569971 +0000 UTC m=+349.730791424" watchObservedRunningTime="2026-07-16 11:09:18.077286688 +0000 UTC m=+350.711508142"
Jul 16 11:11:38.012721 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:11:38.012679    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-cb66496d-mfk4x"]
Jul 16 11:12:03.034832 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:12:03.034782    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-cb66496d-mfk4x" podUID="335cacdf-3743-4030-ac8f-66723698cab1" containerName="console" containerID="cri-o://ff93884e8de7374e84778327f4d195d03ea5cb51c1cfebfa9e062a7c285539c9" gracePeriod=15
Jul 16 11:12:03.287042 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:12:03.286978    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-cb66496d-mfk4x"
Jul 16 11:12:03.426469 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:12:03.426423    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/335cacdf-3743-4030-ac8f-66723698cab1-console-serving-cert\") pod \"335cacdf-3743-4030-ac8f-66723698cab1\" (UID: \"335cacdf-3743-4030-ac8f-66723698cab1\") "
Jul 16 11:12:03.426694 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:12:03.426493    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/335cacdf-3743-4030-ac8f-66723698cab1-console-config\") pod \"335cacdf-3743-4030-ac8f-66723698cab1\" (UID: \"335cacdf-3743-4030-ac8f-66723698cab1\") "
Jul 16 11:12:03.426694 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:12:03.426534    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6g9sc\" (UniqueName: \"kubernetes.io/projected/335cacdf-3743-4030-ac8f-66723698cab1-kube-api-access-6g9sc\") pod \"335cacdf-3743-4030-ac8f-66723698cab1\" (UID: \"335cacdf-3743-4030-ac8f-66723698cab1\") "
Jul 16 11:12:03.426694 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:12:03.426652    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/335cacdf-3743-4030-ac8f-66723698cab1-trusted-ca-bundle\") pod \"335cacdf-3743-4030-ac8f-66723698cab1\" (UID: \"335cacdf-3743-4030-ac8f-66723698cab1\") "
Jul 16 11:12:03.426865 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:12:03.426702    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/335cacdf-3743-4030-ac8f-66723698cab1-oauth-serving-cert\") pod \"335cacdf-3743-4030-ac8f-66723698cab1\" (UID: \"335cacdf-3743-4030-ac8f-66723698cab1\") "
Jul 16 11:12:03.426865 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:12:03.426720    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/335cacdf-3743-4030-ac8f-66723698cab1-service-ca\") pod \"335cacdf-3743-4030-ac8f-66723698cab1\" (UID: \"335cacdf-3743-4030-ac8f-66723698cab1\") "
Jul 16 11:12:03.426865 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:12:03.426848    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/335cacdf-3743-4030-ac8f-66723698cab1-console-oauth-config\") pod \"335cacdf-3743-4030-ac8f-66723698cab1\" (UID: \"335cacdf-3743-4030-ac8f-66723698cab1\") "
Jul 16 11:12:03.427026 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:12:03.426946    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/335cacdf-3743-4030-ac8f-66723698cab1-console-config" (OuterVolumeSpecName: "console-config") pod "335cacdf-3743-4030-ac8f-66723698cab1" (UID: "335cacdf-3743-4030-ac8f-66723698cab1"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 11:12:03.427082 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:12:03.427056    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/335cacdf-3743-4030-ac8f-66723698cab1-service-ca" (OuterVolumeSpecName: "service-ca") pod "335cacdf-3743-4030-ac8f-66723698cab1" (UID: "335cacdf-3743-4030-ac8f-66723698cab1"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 11:12:03.427135 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:12:03.427076    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/335cacdf-3743-4030-ac8f-66723698cab1-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "335cacdf-3743-4030-ac8f-66723698cab1" (UID: "335cacdf-3743-4030-ac8f-66723698cab1"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 11:12:03.427135 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:12:03.427104    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/335cacdf-3743-4030-ac8f-66723698cab1-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "335cacdf-3743-4030-ac8f-66723698cab1" (UID: "335cacdf-3743-4030-ac8f-66723698cab1"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 11:12:03.427235 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:12:03.427202    2583 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/335cacdf-3743-4030-ac8f-66723698cab1-console-config\") on node \"ip-10-0-129-28.ec2.internal\" DevicePath \"\""
Jul 16 11:12:03.427235 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:12:03.427221    2583 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/335cacdf-3743-4030-ac8f-66723698cab1-trusted-ca-bundle\") on node \"ip-10-0-129-28.ec2.internal\" DevicePath \"\""
Jul 16 11:12:03.427235 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:12:03.427231    2583 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/335cacdf-3743-4030-ac8f-66723698cab1-oauth-serving-cert\") on node \"ip-10-0-129-28.ec2.internal\" DevicePath \"\""
Jul 16 11:12:03.427387 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:12:03.427244    2583 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/335cacdf-3743-4030-ac8f-66723698cab1-service-ca\") on node \"ip-10-0-129-28.ec2.internal\" DevicePath \"\""
Jul 16 11:12:03.428815 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:12:03.428776    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/335cacdf-3743-4030-ac8f-66723698cab1-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "335cacdf-3743-4030-ac8f-66723698cab1" (UID: "335cacdf-3743-4030-ac8f-66723698cab1"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 11:12:03.428815 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:12:03.428801    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/335cacdf-3743-4030-ac8f-66723698cab1-kube-api-access-6g9sc" (OuterVolumeSpecName: "kube-api-access-6g9sc") pod "335cacdf-3743-4030-ac8f-66723698cab1" (UID: "335cacdf-3743-4030-ac8f-66723698cab1"). InnerVolumeSpecName "kube-api-access-6g9sc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 11:12:03.428978 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:12:03.428886    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/335cacdf-3743-4030-ac8f-66723698cab1-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "335cacdf-3743-4030-ac8f-66723698cab1" (UID: "335cacdf-3743-4030-ac8f-66723698cab1"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 11:12:03.528305 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:12:03.528264    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6g9sc\" (UniqueName: \"kubernetes.io/projected/335cacdf-3743-4030-ac8f-66723698cab1-kube-api-access-6g9sc\") on node \"ip-10-0-129-28.ec2.internal\" DevicePath \"\""
Jul 16 11:12:03.528305 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:12:03.528297    2583 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/335cacdf-3743-4030-ac8f-66723698cab1-console-oauth-config\") on node \"ip-10-0-129-28.ec2.internal\" DevicePath \"\""
Jul 16 11:12:03.528305 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:12:03.528308    2583 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/335cacdf-3743-4030-ac8f-66723698cab1-console-serving-cert\") on node \"ip-10-0-129-28.ec2.internal\" DevicePath \"\""
Jul 16 11:12:03.799327 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:12:03.799295    2583 generic.go:356] "Generic (PLEG): container finished" podID="335cacdf-3743-4030-ac8f-66723698cab1" containerID="ff93884e8de7374e84778327f4d195d03ea5cb51c1cfebfa9e062a7c285539c9" exitCode=0
Jul 16 11:12:03.799508 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:12:03.799344    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-cb66496d-mfk4x"
Jul 16 11:12:03.799508 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:12:03.799384    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-cb66496d-mfk4x" event={"ID":"335cacdf-3743-4030-ac8f-66723698cab1","Type":"ContainerDied","Data":"ff93884e8de7374e84778327f4d195d03ea5cb51c1cfebfa9e062a7c285539c9"}
Jul 16 11:12:03.799508 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:12:03.799432    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-cb66496d-mfk4x" event={"ID":"335cacdf-3743-4030-ac8f-66723698cab1","Type":"ContainerDied","Data":"18c3e782fedadddf3eb5118022cf26c8bd3852715a31b318ce8dc67cc4036e78"}
Jul 16 11:12:03.799508 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:12:03.799452    2583 scope.go:117] "RemoveContainer" containerID="ff93884e8de7374e84778327f4d195d03ea5cb51c1cfebfa9e062a7c285539c9"
Jul 16 11:12:03.808359 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:12:03.808341    2583 scope.go:117] "RemoveContainer" containerID="ff93884e8de7374e84778327f4d195d03ea5cb51c1cfebfa9e062a7c285539c9"
Jul 16 11:12:03.808665 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:12:03.808637    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ff93884e8de7374e84778327f4d195d03ea5cb51c1cfebfa9e062a7c285539c9\": container with ID starting with ff93884e8de7374e84778327f4d195d03ea5cb51c1cfebfa9e062a7c285539c9 not found: ID does not exist" containerID="ff93884e8de7374e84778327f4d195d03ea5cb51c1cfebfa9e062a7c285539c9"
Jul 16 11:12:03.808726 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:12:03.808675    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ff93884e8de7374e84778327f4d195d03ea5cb51c1cfebfa9e062a7c285539c9"} err="failed to get container status \"ff93884e8de7374e84778327f4d195d03ea5cb51c1cfebfa9e062a7c285539c9\": rpc error: code = NotFound desc = could not find container \"ff93884e8de7374e84778327f4d195d03ea5cb51c1cfebfa9e062a7c285539c9\": container with ID starting with ff93884e8de7374e84778327f4d195d03ea5cb51c1cfebfa9e062a7c285539c9 not found: ID does not exist"
Jul 16 11:12:03.822292 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:12:03.822262    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-cb66496d-mfk4x"]
Jul 16 11:12:03.825984 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:12:03.825957    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-cb66496d-mfk4x"]
Jul 16 11:12:05.228694 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:12:05.228663    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="335cacdf-3743-4030-ac8f-66723698cab1" path="/var/lib/kubelet/pods/335cacdf-3743-4030-ac8f-66723698cab1/volumes"
Jul 16 11:12:22.537755 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:12:22.537664    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7f7784fc54-b2t98"]
Jul 16 11:12:22.538259 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:12:22.537984    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="335cacdf-3743-4030-ac8f-66723698cab1" containerName="console"
Jul 16 11:12:22.538259 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:12:22.537995    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="335cacdf-3743-4030-ac8f-66723698cab1" containerName="console"
Jul 16 11:12:22.538259 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:12:22.538066    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="335cacdf-3743-4030-ac8f-66723698cab1" containerName="console"
Jul 16 11:12:22.540773 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:12:22.540751    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7f7784fc54-b2t98"
Jul 16 11:12:22.543532 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:12:22.543503    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 16 11:12:22.543687 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:12:22.543513    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-6w6j4\""
Jul 16 11:12:22.544393 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:12:22.544376    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 16 11:12:22.559068 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:12:22.559039    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7f7784fc54-b2t98"]
Jul 16 11:12:22.593658 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:12:22.593592    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w6hp5\" (UniqueName: \"kubernetes.io/projected/f69c6b7a-252c-40ec-b710-d8845c49ed0d-kube-api-access-w6hp5\") pod \"maas-controller-7f7784fc54-b2t98\" (UID: \"f69c6b7a-252c-40ec-b710-d8845c49ed0d\") " pod="opendatahub/maas-controller-7f7784fc54-b2t98"
Jul 16 11:12:22.593658 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:12:22.593661    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f69c6b7a-252c-40ec-b710-d8845c49ed0d-webhook-cert\") pod \"maas-controller-7f7784fc54-b2t98\" (UID: \"f69c6b7a-252c-40ec-b710-d8845c49ed0d\") " pod="opendatahub/maas-controller-7f7784fc54-b2t98"
Jul 16 11:12:22.694692 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:12:22.694657    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w6hp5\" (UniqueName: \"kubernetes.io/projected/f69c6b7a-252c-40ec-b710-d8845c49ed0d-kube-api-access-w6hp5\") pod \"maas-controller-7f7784fc54-b2t98\" (UID: \"f69c6b7a-252c-40ec-b710-d8845c49ed0d\") " pod="opendatahub/maas-controller-7f7784fc54-b2t98"
Jul 16 11:12:22.694692 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:12:22.694693    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f69c6b7a-252c-40ec-b710-d8845c49ed0d-webhook-cert\") pod \"maas-controller-7f7784fc54-b2t98\" (UID: \"f69c6b7a-252c-40ec-b710-d8845c49ed0d\") " pod="opendatahub/maas-controller-7f7784fc54-b2t98"
Jul 16 11:12:22.694883 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:12:22.694856    2583 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found
Jul 16 11:12:22.694959 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:12:22.694948    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f69c6b7a-252c-40ec-b710-d8845c49ed0d-webhook-cert podName:f69c6b7a-252c-40ec-b710-d8845c49ed0d nodeName:}" failed. No retries permitted until 2026-07-16 11:12:23.194926739 +0000 UTC m=+535.829148184 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/f69c6b7a-252c-40ec-b710-d8845c49ed0d-webhook-cert") pod "maas-controller-7f7784fc54-b2t98" (UID: "f69c6b7a-252c-40ec-b710-d8845c49ed0d") : secret "maas-controller-webhook-cert" not found
Jul 16 11:12:22.719363 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:12:22.719327    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w6hp5\" (UniqueName: \"kubernetes.io/projected/f69c6b7a-252c-40ec-b710-d8845c49ed0d-kube-api-access-w6hp5\") pod \"maas-controller-7f7784fc54-b2t98\" (UID: \"f69c6b7a-252c-40ec-b710-d8845c49ed0d\") " pod="opendatahub/maas-controller-7f7784fc54-b2t98"
Jul 16 11:12:22.746859 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:12:22.746826    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-98cff6ff4-ttqwj"]
Jul 16 11:12:22.750138 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:12:22.750113    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-98cff6ff4-ttqwj"
Jul 16 11:12:22.761891 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:12:22.761863    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-98cff6ff4-ttqwj"]
Jul 16 11:12:22.796019 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:12:22.795929    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c07e96f5-8869-4edb-989c-31aa50a61a99-webhook-cert\") pod \"maas-controller-98cff6ff4-ttqwj\" (UID: \"c07e96f5-8869-4edb-989c-31aa50a61a99\") " pod="opendatahub/maas-controller-98cff6ff4-ttqwj"
Jul 16 11:12:22.796019 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:12:22.796001    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xc8lc\" (UniqueName: \"kubernetes.io/projected/c07e96f5-8869-4edb-989c-31aa50a61a99-kube-api-access-xc8lc\") pod \"maas-controller-98cff6ff4-ttqwj\" (UID: \"c07e96f5-8869-4edb-989c-31aa50a61a99\") " pod="opendatahub/maas-controller-98cff6ff4-ttqwj"
Jul 16 11:12:22.897123 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:12:22.897087    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c07e96f5-8869-4edb-989c-31aa50a61a99-webhook-cert\") pod \"maas-controller-98cff6ff4-ttqwj\" (UID: \"c07e96f5-8869-4edb-989c-31aa50a61a99\") " pod="opendatahub/maas-controller-98cff6ff4-ttqwj"
Jul 16 11:12:22.897296 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:12:22.897167    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xc8lc\" (UniqueName: \"kubernetes.io/projected/c07e96f5-8869-4edb-989c-31aa50a61a99-kube-api-access-xc8lc\") pod \"maas-controller-98cff6ff4-ttqwj\" (UID: \"c07e96f5-8869-4edb-989c-31aa50a61a99\") " pod="opendatahub/maas-controller-98cff6ff4-ttqwj"
Jul 16 11:12:22.899624 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:12:22.899563    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c07e96f5-8869-4edb-989c-31aa50a61a99-webhook-cert\") pod \"maas-controller-98cff6ff4-ttqwj\" (UID: \"c07e96f5-8869-4edb-989c-31aa50a61a99\") " pod="opendatahub/maas-controller-98cff6ff4-ttqwj"
Jul 16 11:12:22.907788 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:12:22.907759    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xc8lc\" (UniqueName: \"kubernetes.io/projected/c07e96f5-8869-4edb-989c-31aa50a61a99-kube-api-access-xc8lc\") pod \"maas-controller-98cff6ff4-ttqwj\" (UID: \"c07e96f5-8869-4edb-989c-31aa50a61a99\") " pod="opendatahub/maas-controller-98cff6ff4-ttqwj"
Jul 16 11:12:23.061543 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:12:23.061436    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-98cff6ff4-ttqwj"
Jul 16 11:12:23.199455 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:12:23.199422    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f69c6b7a-252c-40ec-b710-d8845c49ed0d-webhook-cert\") pod \"maas-controller-7f7784fc54-b2t98\" (UID: \"f69c6b7a-252c-40ec-b710-d8845c49ed0d\") " pod="opendatahub/maas-controller-7f7784fc54-b2t98"
Jul 16 11:12:23.201949 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:12:23.201929    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f69c6b7a-252c-40ec-b710-d8845c49ed0d-webhook-cert\") pod \"maas-controller-7f7784fc54-b2t98\" (UID: \"f69c6b7a-252c-40ec-b710-d8845c49ed0d\") " pod="opendatahub/maas-controller-7f7784fc54-b2t98"
Jul 16 11:12:23.206115 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:12:23.206082    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-98cff6ff4-ttqwj"]
Jul 16 11:12:23.208350 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:12:23.208319    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc07e96f5_8869_4edb_989c_31aa50a61a99.slice/crio-467fdb8e85d60094f2a016a9036cdcdebd714131d31ac65fd04f04074cd08acd WatchSource:0}: Error finding container 467fdb8e85d60094f2a016a9036cdcdebd714131d31ac65fd04f04074cd08acd: Status 404 returned error can't find the container with id 467fdb8e85d60094f2a016a9036cdcdebd714131d31ac65fd04f04074cd08acd
Jul 16 11:12:23.451522 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:12:23.451484    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7f7784fc54-b2t98"
Jul 16 11:12:23.590262 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:12:23.590230    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7f7784fc54-b2t98"]
Jul 16 11:12:23.592925 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:12:23.592895    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf69c6b7a_252c_40ec_b710_d8845c49ed0d.slice/crio-b88adb37b0d82e566b041d9265be0e4e3caa59e58bde1984e84b2558f26b9b1d WatchSource:0}: Error finding container b88adb37b0d82e566b041d9265be0e4e3caa59e58bde1984e84b2558f26b9b1d: Status 404 returned error can't find the container with id b88adb37b0d82e566b041d9265be0e4e3caa59e58bde1984e84b2558f26b9b1d
Jul 16 11:12:23.893656 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:12:23.893538    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-98cff6ff4-ttqwj" event={"ID":"c07e96f5-8869-4edb-989c-31aa50a61a99","Type":"ContainerStarted","Data":"467fdb8e85d60094f2a016a9036cdcdebd714131d31ac65fd04f04074cd08acd"}
Jul 16 11:12:23.895762 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:12:23.895729    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7f7784fc54-b2t98" event={"ID":"f69c6b7a-252c-40ec-b710-d8845c49ed0d","Type":"ContainerStarted","Data":"b88adb37b0d82e566b041d9265be0e4e3caa59e58bde1984e84b2558f26b9b1d"}
Jul 16 11:12:26.911222 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:12:26.911175    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7f7784fc54-b2t98" event={"ID":"f69c6b7a-252c-40ec-b710-d8845c49ed0d","Type":"ContainerStarted","Data":"088c3d30690e87759ba53775c23c2749901a6e7a72c3a305be751ad82afcc09f"}
Jul 16 11:12:26.911708 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:12:26.911256    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7f7784fc54-b2t98"
Jul 16 11:12:26.912713 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:12:26.912688    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-98cff6ff4-ttqwj" event={"ID":"c07e96f5-8869-4edb-989c-31aa50a61a99","Type":"ContainerStarted","Data":"d3c3b6cbd060e0b671e0ae95ebdbbf9eb6243f42533caf7246ab0e93b5eb9523"}
Jul 16 11:12:26.912835 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:12:26.912823    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-98cff6ff4-ttqwj"
Jul 16 11:12:26.937543 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:12:26.937485    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-7f7784fc54-b2t98" podStartSLOduration=2.57949619 podStartE2EDuration="4.937468218s" totalImagesPullingTime="2.357972028s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 11:12:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 11:12:26.935930561 +0000 UTC m=+539.570152038" watchObservedRunningTime="2026-07-16 11:12:26.937468218 +0000 UTC m=+539.571689672"
Jul 16 11:12:26.956968 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:12:26.956908    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-98cff6ff4-ttqwj" podStartSLOduration=2.224684721 podStartE2EDuration="4.95689198s" totalImagesPullingTime="2.732207259s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 11:12:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 11:12:26.955268921 +0000 UTC m=+539.589490376" watchObservedRunningTime="2026-07-16 11:12:26.95689198 +0000 UTC m=+539.591113433"
Jul 16 11:12:37.919964 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:12:37.919923    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-98cff6ff4-ttqwj"
Jul 16 11:12:37.920480 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:12:37.920102    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7f7784fc54-b2t98"
Jul 16 11:12:37.988871 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:12:37.988828    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7f7784fc54-b2t98"]
Jul 16 11:12:37.989068 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:12:37.989025    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-7f7784fc54-b2t98" podUID="f69c6b7a-252c-40ec-b710-d8845c49ed0d" containerName="manager" containerID="cri-o://088c3d30690e87759ba53775c23c2749901a6e7a72c3a305be751ad82afcc09f" gracePeriod=30
Jul 16 11:12:38.240230 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:12:38.240200    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7f7784fc54-b2t98"
Jul 16 11:12:38.341552 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:12:38.341510    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f69c6b7a-252c-40ec-b710-d8845c49ed0d-webhook-cert\") pod \"f69c6b7a-252c-40ec-b710-d8845c49ed0d\" (UID: \"f69c6b7a-252c-40ec-b710-d8845c49ed0d\") "
Jul 16 11:12:38.341768 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:12:38.341638    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w6hp5\" (UniqueName: \"kubernetes.io/projected/f69c6b7a-252c-40ec-b710-d8845c49ed0d-kube-api-access-w6hp5\") pod \"f69c6b7a-252c-40ec-b710-d8845c49ed0d\" (UID: \"f69c6b7a-252c-40ec-b710-d8845c49ed0d\") "
Jul 16 11:12:38.343785 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:12:38.343757    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f69c6b7a-252c-40ec-b710-d8845c49ed0d-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "f69c6b7a-252c-40ec-b710-d8845c49ed0d" (UID: "f69c6b7a-252c-40ec-b710-d8845c49ed0d"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 11:12:38.343898 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:12:38.343876    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f69c6b7a-252c-40ec-b710-d8845c49ed0d-kube-api-access-w6hp5" (OuterVolumeSpecName: "kube-api-access-w6hp5") pod "f69c6b7a-252c-40ec-b710-d8845c49ed0d" (UID: "f69c6b7a-252c-40ec-b710-d8845c49ed0d"). InnerVolumeSpecName "kube-api-access-w6hp5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 11:12:38.442667 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:12:38.442591    2583 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f69c6b7a-252c-40ec-b710-d8845c49ed0d-webhook-cert\") on node \"ip-10-0-129-28.ec2.internal\" DevicePath \"\""
Jul 16 11:12:38.442667 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:12:38.442662    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w6hp5\" (UniqueName: \"kubernetes.io/projected/f69c6b7a-252c-40ec-b710-d8845c49ed0d-kube-api-access-w6hp5\") on node \"ip-10-0-129-28.ec2.internal\" DevicePath \"\""
Jul 16 11:12:38.975696 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:12:38.975654    2583 generic.go:356] "Generic (PLEG): container finished" podID="f69c6b7a-252c-40ec-b710-d8845c49ed0d" containerID="088c3d30690e87759ba53775c23c2749901a6e7a72c3a305be751ad82afcc09f" exitCode=0
Jul 16 11:12:38.975696 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:12:38.975699    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7f7784fc54-b2t98" event={"ID":"f69c6b7a-252c-40ec-b710-d8845c49ed0d","Type":"ContainerDied","Data":"088c3d30690e87759ba53775c23c2749901a6e7a72c3a305be751ad82afcc09f"}
Jul 16 11:12:38.976231 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:12:38.975738    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7f7784fc54-b2t98" event={"ID":"f69c6b7a-252c-40ec-b710-d8845c49ed0d","Type":"ContainerDied","Data":"b88adb37b0d82e566b041d9265be0e4e3caa59e58bde1984e84b2558f26b9b1d"}
Jul 16 11:12:38.976231 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:12:38.975759    2583 scope.go:117] "RemoveContainer" containerID="088c3d30690e87759ba53775c23c2749901a6e7a72c3a305be751ad82afcc09f"
Jul 16 11:12:38.976231 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:12:38.975699    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7f7784fc54-b2t98"
Jul 16 11:12:38.984496 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:12:38.984478    2583 scope.go:117] "RemoveContainer" containerID="088c3d30690e87759ba53775c23c2749901a6e7a72c3a305be751ad82afcc09f"
Jul 16 11:12:38.984874 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:12:38.984844    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"088c3d30690e87759ba53775c23c2749901a6e7a72c3a305be751ad82afcc09f\": container with ID starting with 088c3d30690e87759ba53775c23c2749901a6e7a72c3a305be751ad82afcc09f not found: ID does not exist" containerID="088c3d30690e87759ba53775c23c2749901a6e7a72c3a305be751ad82afcc09f"
Jul 16 11:12:38.984960 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:12:38.984880    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"088c3d30690e87759ba53775c23c2749901a6e7a72c3a305be751ad82afcc09f"} err="failed to get container status \"088c3d30690e87759ba53775c23c2749901a6e7a72c3a305be751ad82afcc09f\": rpc error: code = NotFound desc = could not find container \"088c3d30690e87759ba53775c23c2749901a6e7a72c3a305be751ad82afcc09f\": container with ID starting with 088c3d30690e87759ba53775c23c2749901a6e7a72c3a305be751ad82afcc09f not found: ID does not exist"
Jul 16 11:12:38.998173 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:12:38.998136    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7f7784fc54-b2t98"]
Jul 16 11:12:39.003153 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:12:39.003124    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-7f7784fc54-b2t98"]
Jul 16 11:12:39.228790 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:12:39.228715    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f69c6b7a-252c-40ec-b710-d8845c49ed0d" path="/var/lib/kubelet/pods/f69c6b7a-252c-40ec-b710-d8845c49ed0d/volumes"
Jul 16 11:12:58.123332 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:12:58.123298    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-77c48f86df-lp76j"]
Jul 16 11:12:58.123888 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:12:58.123647    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f69c6b7a-252c-40ec-b710-d8845c49ed0d" containerName="manager"
Jul 16 11:12:58.123888 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:12:58.123658    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="f69c6b7a-252c-40ec-b710-d8845c49ed0d" containerName="manager"
Jul 16 11:12:58.123888 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:12:58.123753    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="f69c6b7a-252c-40ec-b710-d8845c49ed0d" containerName="manager"
Jul 16 11:12:58.126912 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:12:58.126888    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-77c48f86df-lp76j"
Jul 16 11:12:58.129370 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:12:58.129346    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Jul 16 11:12:58.129950 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:12:58.129930    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 16 11:12:58.130346 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:12:58.130328    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-wmpqk\""
Jul 16 11:12:58.130417 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:12:58.130333    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 16 11:12:58.139498 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:12:58.139470    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-77c48f86df-lp76j"]
Jul 16 11:12:58.319401 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:12:58.319357    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/35093d76-01dc-4292-9bff-d38dae1c7680-openshift-service-ca\") pod \"maas-api-77c48f86df-lp76j\" (UID: \"35093d76-01dc-4292-9bff-d38dae1c7680\") " pod="odh-ai-gateway-infra/maas-api-77c48f86df-lp76j"
Jul 16 11:12:58.319644 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:12:58.319407    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s2fbm\" (UniqueName: \"kubernetes.io/projected/35093d76-01dc-4292-9bff-d38dae1c7680-kube-api-access-s2fbm\") pod \"maas-api-77c48f86df-lp76j\" (UID: \"35093d76-01dc-4292-9bff-d38dae1c7680\") " pod="odh-ai-gateway-infra/maas-api-77c48f86df-lp76j"
Jul 16 11:12:58.319644 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:12:58.319473    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/35093d76-01dc-4292-9bff-d38dae1c7680-maas-api-tls\") pod \"maas-api-77c48f86df-lp76j\" (UID: \"35093d76-01dc-4292-9bff-d38dae1c7680\") " pod="odh-ai-gateway-infra/maas-api-77c48f86df-lp76j"
Jul 16 11:12:58.420897 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:12:58.420864    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/35093d76-01dc-4292-9bff-d38dae1c7680-openshift-service-ca\") pod \"maas-api-77c48f86df-lp76j\" (UID: \"35093d76-01dc-4292-9bff-d38dae1c7680\") " pod="odh-ai-gateway-infra/maas-api-77c48f86df-lp76j"
Jul 16 11:12:58.421109 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:12:58.420907    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s2fbm\" (UniqueName: \"kubernetes.io/projected/35093d76-01dc-4292-9bff-d38dae1c7680-kube-api-access-s2fbm\") pod \"maas-api-77c48f86df-lp76j\" (UID: \"35093d76-01dc-4292-9bff-d38dae1c7680\") " pod="odh-ai-gateway-infra/maas-api-77c48f86df-lp76j"
Jul 16 11:12:58.421109 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:12:58.420943    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/35093d76-01dc-4292-9bff-d38dae1c7680-maas-api-tls\") pod \"maas-api-77c48f86df-lp76j\" (UID: \"35093d76-01dc-4292-9bff-d38dae1c7680\") " pod="odh-ai-gateway-infra/maas-api-77c48f86df-lp76j"
Jul 16 11:12:58.421665 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:12:58.421641    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/35093d76-01dc-4292-9bff-d38dae1c7680-openshift-service-ca\") pod \"maas-api-77c48f86df-lp76j\" (UID: \"35093d76-01dc-4292-9bff-d38dae1c7680\") " pod="odh-ai-gateway-infra/maas-api-77c48f86df-lp76j"
Jul 16 11:12:58.423610 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:12:58.423577    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/35093d76-01dc-4292-9bff-d38dae1c7680-maas-api-tls\") pod \"maas-api-77c48f86df-lp76j\" (UID: \"35093d76-01dc-4292-9bff-d38dae1c7680\") " pod="odh-ai-gateway-infra/maas-api-77c48f86df-lp76j"
Jul 16 11:12:58.448609 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:12:58.448565    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s2fbm\" (UniqueName: \"kubernetes.io/projected/35093d76-01dc-4292-9bff-d38dae1c7680-kube-api-access-s2fbm\") pod \"maas-api-77c48f86df-lp76j\" (UID: \"35093d76-01dc-4292-9bff-d38dae1c7680\") " pod="odh-ai-gateway-infra/maas-api-77c48f86df-lp76j"
Jul 16 11:12:58.738863 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:12:58.738771    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-77c48f86df-lp76j"
Jul 16 11:12:58.892447 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:12:58.892412    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-77c48f86df-lp76j"]
Jul 16 11:12:58.896372 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:12:58.896335    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35093d76_01dc_4292_9bff_d38dae1c7680.slice/crio-1542e34b09ff368d6664bbca46bb44f1bec243748b669b0f44de5c125d76f4f8 WatchSource:0}: Error finding container 1542e34b09ff368d6664bbca46bb44f1bec243748b669b0f44de5c125d76f4f8: Status 404 returned error can't find the container with id 1542e34b09ff368d6664bbca46bb44f1bec243748b669b0f44de5c125d76f4f8
Jul 16 11:12:59.068356 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:12:59.068270    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-77c48f86df-lp76j" event={"ID":"35093d76-01dc-4292-9bff-d38dae1c7680","Type":"ContainerStarted","Data":"1542e34b09ff368d6664bbca46bb44f1bec243748b669b0f44de5c125d76f4f8"}
Jul 16 11:13:01.079706 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:13:01.079651    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-77c48f86df-lp76j" event={"ID":"35093d76-01dc-4292-9bff-d38dae1c7680","Type":"ContainerStarted","Data":"d86d940ca60ae897a00d2acb5d2a4fd39891afa119a5e8f4072c01b9c477e5f8"}
Jul 16 11:13:01.080094 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:13:01.079753    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-77c48f86df-lp76j"
Jul 16 11:13:01.109427 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:13:01.109368    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-77c48f86df-lp76j" podStartSLOduration=1.058301173 podStartE2EDuration="3.109349985s" totalImagesPullingTime="2.051048812s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 11:12:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 11:13:01.107247503 +0000 UTC m=+573.741468983" watchObservedRunningTime="2026-07-16 11:13:01.109349985 +0000 UTC m=+573.743571439"
Jul 16 11:13:07.089749 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:13:07.089716    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-77c48f86df-lp76j"
Jul 16 11:13:14.415552 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:13:14.415516    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-p4mpc"]
Jul 16 11:13:14.418979 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:13:14.418960    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-p4mpc"
Jul 16 11:13:14.421543 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:13:14.421518    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 16 11:13:14.422592 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:13:14.422570    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 16 11:13:14.422673 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:13:14.422633    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-xzgtr\""
Jul 16 11:13:14.422719 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:13:14.422570    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Jul 16 11:13:14.428989 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:13:14.428961    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-p4mpc"]
Jul 16 11:13:14.462797 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:13:14.462760    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/179c766b-106b-4a80-a469-31ceb07ce46b-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-p4mpc\" (UID: \"179c766b-106b-4a80-a469-31ceb07ce46b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-p4mpc"
Jul 16 11:13:14.462797 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:13:14.462798    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/179c766b-106b-4a80-a469-31ceb07ce46b-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-p4mpc\" (UID: \"179c766b-106b-4a80-a469-31ceb07ce46b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-p4mpc"
Jul 16 11:13:14.463010 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:13:14.462824    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/179c766b-106b-4a80-a469-31ceb07ce46b-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-p4mpc\" (UID: \"179c766b-106b-4a80-a469-31ceb07ce46b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-p4mpc"
Jul 16 11:13:14.463010 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:13:14.462849    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xgwld\" (UniqueName: \"kubernetes.io/projected/179c766b-106b-4a80-a469-31ceb07ce46b-kube-api-access-xgwld\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-p4mpc\" (UID: \"179c766b-106b-4a80-a469-31ceb07ce46b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-p4mpc"
Jul 16 11:13:14.463010 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:13:14.462901    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/179c766b-106b-4a80-a469-31ceb07ce46b-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-p4mpc\" (UID: \"179c766b-106b-4a80-a469-31ceb07ce46b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-p4mpc"
Jul 16 11:13:14.463010 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:13:14.462923    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/179c766b-106b-4a80-a469-31ceb07ce46b-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-p4mpc\" (UID: \"179c766b-106b-4a80-a469-31ceb07ce46b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-p4mpc"
Jul 16 11:13:14.563801 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:13:14.563763    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/179c766b-106b-4a80-a469-31ceb07ce46b-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-p4mpc\" (UID: \"179c766b-106b-4a80-a469-31ceb07ce46b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-p4mpc"
Jul 16 11:13:14.563801 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:13:14.563799    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/179c766b-106b-4a80-a469-31ceb07ce46b-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-p4mpc\" (UID: \"179c766b-106b-4a80-a469-31ceb07ce46b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-p4mpc"
Jul 16 11:13:14.564031 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:13:14.563818    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/179c766b-106b-4a80-a469-31ceb07ce46b-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-p4mpc\" (UID: \"179c766b-106b-4a80-a469-31ceb07ce46b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-p4mpc"
Jul 16 11:13:14.564031 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:13:14.563835    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xgwld\" (UniqueName: \"kubernetes.io/projected/179c766b-106b-4a80-a469-31ceb07ce46b-kube-api-access-xgwld\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-p4mpc\" (UID: \"179c766b-106b-4a80-a469-31ceb07ce46b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-p4mpc"
Jul 16 11:13:14.564031 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:13:14.563980    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/179c766b-106b-4a80-a469-31ceb07ce46b-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-p4mpc\" (UID: \"179c766b-106b-4a80-a469-31ceb07ce46b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-p4mpc"
Jul 16 11:13:14.564180 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:13:14.564027    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/179c766b-106b-4a80-a469-31ceb07ce46b-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-p4mpc\" (UID: \"179c766b-106b-4a80-a469-31ceb07ce46b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-p4mpc"
Jul 16 11:13:14.564231 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:13:14.564183    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/179c766b-106b-4a80-a469-31ceb07ce46b-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-p4mpc\" (UID: \"179c766b-106b-4a80-a469-31ceb07ce46b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-p4mpc"
Jul 16 11:13:14.564284 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:13:14.564231    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/179c766b-106b-4a80-a469-31ceb07ce46b-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-p4mpc\" (UID: \"179c766b-106b-4a80-a469-31ceb07ce46b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-p4mpc"
Jul 16 11:13:14.564465 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:13:14.564427    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/179c766b-106b-4a80-a469-31ceb07ce46b-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-p4mpc\" (UID: \"179c766b-106b-4a80-a469-31ceb07ce46b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-p4mpc"
Jul 16 11:13:14.566278 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:13:14.566252    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/179c766b-106b-4a80-a469-31ceb07ce46b-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-p4mpc\" (UID: \"179c766b-106b-4a80-a469-31ceb07ce46b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-p4mpc"
Jul 16 11:13:14.566403 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:13:14.566384    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/179c766b-106b-4a80-a469-31ceb07ce46b-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-p4mpc\" (UID: \"179c766b-106b-4a80-a469-31ceb07ce46b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-p4mpc"
Jul 16 11:13:14.578124 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:13:14.578096    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xgwld\" (UniqueName: \"kubernetes.io/projected/179c766b-106b-4a80-a469-31ceb07ce46b-kube-api-access-xgwld\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-p4mpc\" (UID: \"179c766b-106b-4a80-a469-31ceb07ce46b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-p4mpc"
Jul 16 11:13:14.730623 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:13:14.730518    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-p4mpc"
Jul 16 11:13:14.871006 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:13:14.870978    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-p4mpc"]
Jul 16 11:13:14.873187 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:13:14.873157    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod179c766b_106b_4a80_a469_31ceb07ce46b.slice/crio-857c2494e5f8b2077d9ac177adba4ea29ffd36983ac046054a7a878ef23b609d WatchSource:0}: Error finding container 857c2494e5f8b2077d9ac177adba4ea29ffd36983ac046054a7a878ef23b609d: Status 404 returned error can't find the container with id 857c2494e5f8b2077d9ac177adba4ea29ffd36983ac046054a7a878ef23b609d
Jul 16 11:13:15.146673 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:13:15.146629    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-p4mpc" event={"ID":"179c766b-106b-4a80-a469-31ceb07ce46b","Type":"ContainerStarted","Data":"857c2494e5f8b2077d9ac177adba4ea29ffd36983ac046054a7a878ef23b609d"}
Jul 16 11:13:16.153227 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:13:16.153192    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-p4mpc" event={"ID":"179c766b-106b-4a80-a469-31ceb07ce46b","Type":"ContainerStarted","Data":"d5db1280689016a76096c00c9487c4d8cc97f3bd7c8bce1ef1dae616fb1ee612"}
Jul 16 11:13:16.175027 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:13:16.174972    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-p4mpc" podStartSLOduration=0.985601086 podStartE2EDuration="2.174953659s" totalImagesPullingTime="1.189352573s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 11:13:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 11:13:16.173184818 +0000 UTC m=+588.807406297" watchObservedRunningTime="2026-07-16 11:13:16.174953659 +0000 UTC m=+588.809175112"
Jul 16 11:13:24.730884 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:13:24.730846    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-p4mpc"
Jul 16 11:13:24.731285 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:13:24.730911    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-p4mpc"
Jul 16 11:13:24.743749 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:13:24.743722    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-p4mpc"
Jul 16 11:13:25.206412 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:13:25.206372    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-p4mpc"
Jul 16 11:13:29.134422 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:13:29.134397    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-9lm5s_f64f15a3-5a9f-4737-bc4f-73d5ec311e41/console-operator/1.log"
Jul 16 11:13:29.135666 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:13:29.135644    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-9lm5s_f64f15a3-5a9f-4737-bc4f-73d5ec311e41/console-operator/1.log"
Jul 16 11:13:29.140276 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:13:29.140257    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2db6t_69aed169-e8af-429d-99c3-3b20be64e351/ovn-acl-logging/0.log"
Jul 16 11:13:29.141468 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:13:29.141450    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2db6t_69aed169-e8af-429d-99c3-3b20be64e351/ovn-acl-logging/0.log"
Jul 16 11:13:40.809499 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:13:40.809459    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbdtxd5"]
Jul 16 11:13:40.812876 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:13:40.812857    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbdtxd5"
Jul 16 11:13:40.816642 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:13:40.816616    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Jul 16 11:13:40.829332 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:13:40.829303    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbdtxd5"]
Jul 16 11:13:40.904204 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:13:40.904160    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x4p4n\" (UniqueName: \"kubernetes.io/projected/ae994389-c011-4f3e-b27c-1cded9760559-kube-api-access-x4p4n\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbdtxd5\" (UID: \"ae994389-c011-4f3e-b27c-1cded9760559\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbdtxd5"
Jul 16 11:13:40.904377 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:13:40.904207    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ae994389-c011-4f3e-b27c-1cded9760559-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbdtxd5\" (UID: \"ae994389-c011-4f3e-b27c-1cded9760559\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbdtxd5"
Jul 16 11:13:40.904377 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:13:40.904288    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ae994389-c011-4f3e-b27c-1cded9760559-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbdtxd5\" (UID: \"ae994389-c011-4f3e-b27c-1cded9760559\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbdtxd5"
Jul 16 11:13:40.904377 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:13:40.904318    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ae994389-c011-4f3e-b27c-1cded9760559-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbdtxd5\" (UID: \"ae994389-c011-4f3e-b27c-1cded9760559\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbdtxd5"
Jul 16 11:13:40.904377 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:13:40.904365    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ae994389-c011-4f3e-b27c-1cded9760559-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbdtxd5\" (UID: \"ae994389-c011-4f3e-b27c-1cded9760559\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbdtxd5"
Jul 16 11:13:40.904558 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:13:40.904384    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ae994389-c011-4f3e-b27c-1cded9760559-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbdtxd5\" (UID: \"ae994389-c011-4f3e-b27c-1cded9760559\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbdtxd5"
Jul 16 11:13:41.004897 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:13:41.004858    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ae994389-c011-4f3e-b27c-1cded9760559-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbdtxd5\" (UID: \"ae994389-c011-4f3e-b27c-1cded9760559\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbdtxd5"
Jul 16 11:13:41.005105 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:13:41.004934    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ae994389-c011-4f3e-b27c-1cded9760559-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbdtxd5\" (UID: \"ae994389-c011-4f3e-b27c-1cded9760559\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbdtxd5"
Jul 16 11:13:41.005105 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:13:41.004959    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ae994389-c011-4f3e-b27c-1cded9760559-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbdtxd5\" (UID: \"ae994389-c011-4f3e-b27c-1cded9760559\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbdtxd5"
Jul 16 11:13:41.005105 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:13:41.005031    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x4p4n\" (UniqueName: \"kubernetes.io/projected/ae994389-c011-4f3e-b27c-1cded9760559-kube-api-access-x4p4n\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbdtxd5\" (UID: \"ae994389-c011-4f3e-b27c-1cded9760559\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbdtxd5"
Jul 16 11:13:41.005105 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:13:41.005059    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ae994389-c011-4f3e-b27c-1cded9760559-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbdtxd5\" (UID: \"ae994389-c011-4f3e-b27c-1cded9760559\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbdtxd5"
Jul 16 11:13:41.005105 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:13:41.005103    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ae994389-c011-4f3e-b27c-1cded9760559-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbdtxd5\" (UID: \"ae994389-c011-4f3e-b27c-1cded9760559\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbdtxd5"
Jul 16 11:13:41.005375 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:13:41.005285    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ae994389-c011-4f3e-b27c-1cded9760559-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbdtxd5\" (UID: \"ae994389-c011-4f3e-b27c-1cded9760559\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbdtxd5"
Jul 16 11:13:41.005375 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:13:41.005304    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ae994389-c011-4f3e-b27c-1cded9760559-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbdtxd5\" (UID: \"ae994389-c011-4f3e-b27c-1cded9760559\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbdtxd5"
Jul 16 11:13:41.005505 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:13:41.005484    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ae994389-c011-4f3e-b27c-1cded9760559-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbdtxd5\" (UID: \"ae994389-c011-4f3e-b27c-1cded9760559\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbdtxd5"
Jul 16 11:13:41.007404 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:13:41.007374    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ae994389-c011-4f3e-b27c-1cded9760559-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbdtxd5\" (UID: \"ae994389-c011-4f3e-b27c-1cded9760559\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbdtxd5"
Jul 16 11:13:41.007591 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:13:41.007569    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ae994389-c011-4f3e-b27c-1cded9760559-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbdtxd5\" (UID: \"ae994389-c011-4f3e-b27c-1cded9760559\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbdtxd5"
Jul 16 11:13:41.017160 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:13:41.017124    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x4p4n\" (UniqueName: \"kubernetes.io/projected/ae994389-c011-4f3e-b27c-1cded9760559-kube-api-access-x4p4n\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbdtxd5\" (UID: \"ae994389-c011-4f3e-b27c-1cded9760559\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbdtxd5"
Jul 16 11:13:41.123895 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:13:41.123799    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbdtxd5"
Jul 16 11:13:41.266151 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:13:41.266127    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbdtxd5"]
Jul 16 11:13:41.268633 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:13:41.268589    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae994389_c011_4f3e_b27c_1cded9760559.slice/crio-2fb647fc2e3f2fcd2e038f5730b32886ccb51c3c10d051cb8df15c5d1cfa2216 WatchSource:0}: Error finding container 2fb647fc2e3f2fcd2e038f5730b32886ccb51c3c10d051cb8df15c5d1cfa2216: Status 404 returned error can't find the container with id 2fb647fc2e3f2fcd2e038f5730b32886ccb51c3c10d051cb8df15c5d1cfa2216
Jul 16 11:13:41.269790 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:13:41.269774    2583 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 16 11:13:42.270914 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:13:42.270871    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbdtxd5" event={"ID":"ae994389-c011-4f3e-b27c-1cded9760559","Type":"ContainerStarted","Data":"ee99686d1752294b2f67bce113ab49f16d8fd6cd55f64a5bfa361ce96205542d"}
Jul 16 11:13:42.271376 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:13:42.270931    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbdtxd5" event={"ID":"ae994389-c011-4f3e-b27c-1cded9760559","Type":"ContainerStarted","Data":"2fb647fc2e3f2fcd2e038f5730b32886ccb51c3c10d051cb8df15c5d1cfa2216"}
Jul 16 11:13:51.124521 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:13:51.124436    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbdtxd5"
Jul 16 11:13:51.124927 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:13:51.124545    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbdtxd5"
Jul 16 11:13:51.137031 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:13:51.136996    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbdtxd5"
Jul 16 11:13:51.175147 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:13:51.175090    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbdtxd5" podStartSLOduration=10.984706835 podStartE2EDuration="11.175071113s" totalImagesPullingTime="190.364278ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 11:13:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 11:13:42.30719879 +0000 UTC m=+614.941420265" watchObservedRunningTime="2026-07-16 11:13:51.175071113 +0000 UTC m=+623.809292567"
Jul 16 11:13:51.323051 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:13:51.323020    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbdtxd5"
Jul 16 11:15:00.277194 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:15:00.277156    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29736675-p49jg"]
Jul 16 11:15:00.280502 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:15:00.280483    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29736675-p49jg"
Jul 16 11:15:00.288189 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:15:00.288169    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-4cj8c\""
Jul 16 11:15:00.289922 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:15:00.289904    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29736675-p49jg"
Jul 16 11:15:00.302052 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:15:00.302024    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29736675-p49jg"]
Jul 16 11:15:00.449213 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:15:00.449180    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29736675-p49jg"]
Jul 16 11:15:00.452463 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:15:00.452435    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd310be22_325c_4a0a_ba64_60024bd4c6eb.slice/crio-d74235ab0c9450b6fbdd80a37af4fc8454ab8519c41aa78322b10ba64067d2f5 WatchSource:0}: Error finding container d74235ab0c9450b6fbdd80a37af4fc8454ab8519c41aa78322b10ba64067d2f5: Status 404 returned error can't find the container with id d74235ab0c9450b6fbdd80a37af4fc8454ab8519c41aa78322b10ba64067d2f5
Jul 16 11:15:00.636650 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:15:00.636551    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29736675-p49jg" event={"ID":"d310be22-325c-4a0a-ba64-60024bd4c6eb","Type":"ContainerStarted","Data":"d74235ab0c9450b6fbdd80a37af4fc8454ab8519c41aa78322b10ba64067d2f5"}
Jul 16 11:15:03.653079 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:15:03.653033    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29736675-p49jg" event={"ID":"d310be22-325c-4a0a-ba64-60024bd4c6eb","Type":"ContainerStarted","Data":"147722f172b143f05d85840a28b81355f94ac25e9d18befe0a3f3622e48a13db"}
Jul 16 11:15:03.670556 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:15:03.670506    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29736675-p49jg" podStartSLOduration=1.195595695 podStartE2EDuration="3.670490809s" totalImagesPullingTime="2.474895114s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 11:15:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 11:15:03.668975279 +0000 UTC m=+696.303196728" watchObservedRunningTime="2026-07-16 11:15:03.670490809 +0000 UTC m=+696.304712262"
Jul 16 11:15:04.659480 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:15:04.659448    2583 generic.go:356] "Generic (PLEG): container finished" podID="d310be22-325c-4a0a-ba64-60024bd4c6eb" containerID="147722f172b143f05d85840a28b81355f94ac25e9d18befe0a3f3622e48a13db" exitCode=0
Jul 16 11:15:04.659900 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:15:04.659509    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29736675-p49jg" event={"ID":"d310be22-325c-4a0a-ba64-60024bd4c6eb","Type":"ContainerDied","Data":"147722f172b143f05d85840a28b81355f94ac25e9d18befe0a3f3622e48a13db"}
Jul 16 11:15:05.795690 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:15:05.795665    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29736675-p49jg"
Jul 16 11:15:06.670491 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:15:06.670457    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29736675-p49jg"
Jul 16 11:15:06.670691 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:15:06.670494    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29736675-p49jg" event={"ID":"d310be22-325c-4a0a-ba64-60024bd4c6eb","Type":"ContainerDied","Data":"d74235ab0c9450b6fbdd80a37af4fc8454ab8519c41aa78322b10ba64067d2f5"}
Jul 16 11:15:06.670691 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:15:06.670549    2583 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d74235ab0c9450b6fbdd80a37af4fc8454ab8519c41aa78322b10ba64067d2f5"
Jul 16 11:16:12.511013 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:16:12.510969    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-98cff6ff4-ttqwj"]
Jul 16 11:16:12.511528 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:16:12.511204    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-98cff6ff4-ttqwj" podUID="c07e96f5-8869-4edb-989c-31aa50a61a99" containerName="manager" containerID="cri-o://d3c3b6cbd060e0b671e0ae95ebdbbf9eb6243f42533caf7246ab0e93b5eb9523" gracePeriod=30
Jul 16 11:16:12.765154 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:16:12.765080    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-98cff6ff4-ttqwj"
Jul 16 11:16:12.782817 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:16:12.782777    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xc8lc\" (UniqueName: \"kubernetes.io/projected/c07e96f5-8869-4edb-989c-31aa50a61a99-kube-api-access-xc8lc\") pod \"c07e96f5-8869-4edb-989c-31aa50a61a99\" (UID: \"c07e96f5-8869-4edb-989c-31aa50a61a99\") "
Jul 16 11:16:12.782978 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:16:12.782834    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c07e96f5-8869-4edb-989c-31aa50a61a99-webhook-cert\") pod \"c07e96f5-8869-4edb-989c-31aa50a61a99\" (UID: \"c07e96f5-8869-4edb-989c-31aa50a61a99\") "
Jul 16 11:16:12.785266 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:16:12.785233    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c07e96f5-8869-4edb-989c-31aa50a61a99-kube-api-access-xc8lc" (OuterVolumeSpecName: "kube-api-access-xc8lc") pod "c07e96f5-8869-4edb-989c-31aa50a61a99" (UID: "c07e96f5-8869-4edb-989c-31aa50a61a99"). InnerVolumeSpecName "kube-api-access-xc8lc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 11:16:12.785266 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:16:12.785240    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c07e96f5-8869-4edb-989c-31aa50a61a99-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "c07e96f5-8869-4edb-989c-31aa50a61a99" (UID: "c07e96f5-8869-4edb-989c-31aa50a61a99"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 11:16:12.883642 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:16:12.883607    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xc8lc\" (UniqueName: \"kubernetes.io/projected/c07e96f5-8869-4edb-989c-31aa50a61a99-kube-api-access-xc8lc\") on node \"ip-10-0-129-28.ec2.internal\" DevicePath \"\""
Jul 16 11:16:12.883642 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:16:12.883636    2583 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c07e96f5-8869-4edb-989c-31aa50a61a99-webhook-cert\") on node \"ip-10-0-129-28.ec2.internal\" DevicePath \"\""
Jul 16 11:16:12.994648 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:16:12.994590    2583 generic.go:356] "Generic (PLEG): container finished" podID="c07e96f5-8869-4edb-989c-31aa50a61a99" containerID="d3c3b6cbd060e0b671e0ae95ebdbbf9eb6243f42533caf7246ab0e93b5eb9523" exitCode=0
Jul 16 11:16:12.994648 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:16:12.994650    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-98cff6ff4-ttqwj"
Jul 16 11:16:12.994873 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:16:12.994642    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-98cff6ff4-ttqwj" event={"ID":"c07e96f5-8869-4edb-989c-31aa50a61a99","Type":"ContainerDied","Data":"d3c3b6cbd060e0b671e0ae95ebdbbf9eb6243f42533caf7246ab0e93b5eb9523"}
Jul 16 11:16:12.994873 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:16:12.994772    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-98cff6ff4-ttqwj" event={"ID":"c07e96f5-8869-4edb-989c-31aa50a61a99","Type":"ContainerDied","Data":"467fdb8e85d60094f2a016a9036cdcdebd714131d31ac65fd04f04074cd08acd"}
Jul 16 11:16:12.994873 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:16:12.994796    2583 scope.go:117] "RemoveContainer" containerID="d3c3b6cbd060e0b671e0ae95ebdbbf9eb6243f42533caf7246ab0e93b5eb9523"
Jul 16 11:16:13.003860 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:16:13.003841    2583 scope.go:117] "RemoveContainer" containerID="d3c3b6cbd060e0b671e0ae95ebdbbf9eb6243f42533caf7246ab0e93b5eb9523"
Jul 16 11:16:13.004152 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:16:13.004132    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d3c3b6cbd060e0b671e0ae95ebdbbf9eb6243f42533caf7246ab0e93b5eb9523\": container with ID starting with d3c3b6cbd060e0b671e0ae95ebdbbf9eb6243f42533caf7246ab0e93b5eb9523 not found: ID does not exist" containerID="d3c3b6cbd060e0b671e0ae95ebdbbf9eb6243f42533caf7246ab0e93b5eb9523"
Jul 16 11:16:13.004225 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:16:13.004165    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d3c3b6cbd060e0b671e0ae95ebdbbf9eb6243f42533caf7246ab0e93b5eb9523"} err="failed to get container status \"d3c3b6cbd060e0b671e0ae95ebdbbf9eb6243f42533caf7246ab0e93b5eb9523\": rpc error: code = NotFound desc = could not find container \"d3c3b6cbd060e0b671e0ae95ebdbbf9eb6243f42533caf7246ab0e93b5eb9523\": container with ID starting with d3c3b6cbd060e0b671e0ae95ebdbbf9eb6243f42533caf7246ab0e93b5eb9523 not found: ID does not exist"
Jul 16 11:16:13.017504 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:16:13.017434    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-98cff6ff4-ttqwj"]
Jul 16 11:16:13.019589 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:16:13.019558    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-98cff6ff4-ttqwj"]
Jul 16 11:16:13.228490 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:16:13.228456    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c07e96f5-8869-4edb-989c-31aa50a61a99" path="/var/lib/kubelet/pods/c07e96f5-8869-4edb-989c-31aa50a61a99/volumes"
Jul 16 11:16:13.830451 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:16:13.830417    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-98cff6ff4-mw59h"]
Jul 16 11:16:13.830864 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:16:13.830757    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d310be22-325c-4a0a-ba64-60024bd4c6eb" containerName="cleanup"
Jul 16 11:16:13.830864 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:16:13.830768    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="d310be22-325c-4a0a-ba64-60024bd4c6eb" containerName="cleanup"
Jul 16 11:16:13.830864 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:16:13.830782    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c07e96f5-8869-4edb-989c-31aa50a61a99" containerName="manager"
Jul 16 11:16:13.830864 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:16:13.830788    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="c07e96f5-8869-4edb-989c-31aa50a61a99" containerName="manager"
Jul 16 11:16:13.830864 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:16:13.830846    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="c07e96f5-8869-4edb-989c-31aa50a61a99" containerName="manager"
Jul 16 11:16:13.830864 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:16:13.830853    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="d310be22-325c-4a0a-ba64-60024bd4c6eb" containerName="cleanup"
Jul 16 11:16:13.835352 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:16:13.835331    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-98cff6ff4-mw59h"
Jul 16 11:16:13.837851 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:16:13.837821    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-6w6j4\""
Jul 16 11:16:13.837962 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:16:13.837856    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 16 11:16:13.838770 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:16:13.838753    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 16 11:16:13.844744 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:16:13.844719    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-98cff6ff4-mw59h"]
Jul 16 11:16:13.891548 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:16:13.891511    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1fdde598-de84-4b7e-8f44-03b0fe2905a5-webhook-cert\") pod \"maas-controller-98cff6ff4-mw59h\" (UID: \"1fdde598-de84-4b7e-8f44-03b0fe2905a5\") " pod="opendatahub/maas-controller-98cff6ff4-mw59h"
Jul 16 11:16:13.891548 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:16:13.891550    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2srxk\" (UniqueName: \"kubernetes.io/projected/1fdde598-de84-4b7e-8f44-03b0fe2905a5-kube-api-access-2srxk\") pod \"maas-controller-98cff6ff4-mw59h\" (UID: \"1fdde598-de84-4b7e-8f44-03b0fe2905a5\") " pod="opendatahub/maas-controller-98cff6ff4-mw59h"
Jul 16 11:16:13.992544 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:16:13.992502    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1fdde598-de84-4b7e-8f44-03b0fe2905a5-webhook-cert\") pod \"maas-controller-98cff6ff4-mw59h\" (UID: \"1fdde598-de84-4b7e-8f44-03b0fe2905a5\") " pod="opendatahub/maas-controller-98cff6ff4-mw59h"
Jul 16 11:16:13.992544 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:16:13.992546    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2srxk\" (UniqueName: \"kubernetes.io/projected/1fdde598-de84-4b7e-8f44-03b0fe2905a5-kube-api-access-2srxk\") pod \"maas-controller-98cff6ff4-mw59h\" (UID: \"1fdde598-de84-4b7e-8f44-03b0fe2905a5\") " pod="opendatahub/maas-controller-98cff6ff4-mw59h"
Jul 16 11:16:13.995083 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:16:13.995055    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1fdde598-de84-4b7e-8f44-03b0fe2905a5-webhook-cert\") pod \"maas-controller-98cff6ff4-mw59h\" (UID: \"1fdde598-de84-4b7e-8f44-03b0fe2905a5\") " pod="opendatahub/maas-controller-98cff6ff4-mw59h"
Jul 16 11:16:14.001841 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:16:14.001814    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2srxk\" (UniqueName: \"kubernetes.io/projected/1fdde598-de84-4b7e-8f44-03b0fe2905a5-kube-api-access-2srxk\") pod \"maas-controller-98cff6ff4-mw59h\" (UID: \"1fdde598-de84-4b7e-8f44-03b0fe2905a5\") " pod="opendatahub/maas-controller-98cff6ff4-mw59h"
Jul 16 11:16:14.145853 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:16:14.145813    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-98cff6ff4-mw59h"
Jul 16 11:16:14.279293 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:16:14.279259    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-98cff6ff4-mw59h"]
Jul 16 11:16:14.283186 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:16:14.283158    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1fdde598_de84_4b7e_8f44_03b0fe2905a5.slice/crio-bf277e36222b030d16e96eed2f900fc5cdd87df542130f5b8b0db50492c3fb1d WatchSource:0}: Error finding container bf277e36222b030d16e96eed2f900fc5cdd87df542130f5b8b0db50492c3fb1d: Status 404 returned error can't find the container with id bf277e36222b030d16e96eed2f900fc5cdd87df542130f5b8b0db50492c3fb1d
Jul 16 11:16:15.006218 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:16:15.006181    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-98cff6ff4-mw59h" event={"ID":"1fdde598-de84-4b7e-8f44-03b0fe2905a5","Type":"ContainerStarted","Data":"56131574c7440ed1190f73642b1376c425aeb14497454cc0454e872e4a7f62f5"}
Jul 16 11:16:15.006700 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:16:15.006232    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-98cff6ff4-mw59h" event={"ID":"1fdde598-de84-4b7e-8f44-03b0fe2905a5","Type":"ContainerStarted","Data":"bf277e36222b030d16e96eed2f900fc5cdd87df542130f5b8b0db50492c3fb1d"}
Jul 16 11:16:15.006700 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:16:15.006264    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-98cff6ff4-mw59h"
Jul 16 11:16:15.024572 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:16:15.024468    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-98cff6ff4-mw59h" podStartSLOduration=1.536227316 podStartE2EDuration="2.024449662s" totalImagesPullingTime="488.222346ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 11:16:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 11:16:15.023350533 +0000 UTC m=+767.657571986" watchObservedRunningTime="2026-07-16 11:16:15.024449662 +0000 UTC m=+767.658671116"
Jul 16 11:16:26.013239 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:16:26.013208    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-98cff6ff4-mw59h"
Jul 16 11:18:29.159892 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:18:29.159820    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-9lm5s_f64f15a3-5a9f-4737-bc4f-73d5ec311e41/console-operator/1.log"
Jul 16 11:18:29.162870 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:18:29.162849    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-9lm5s_f64f15a3-5a9f-4737-bc4f-73d5ec311e41/console-operator/1.log"
Jul 16 11:18:29.166429 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:18:29.166409    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2db6t_69aed169-e8af-429d-99c3-3b20be64e351/ovn-acl-logging/0.log"
Jul 16 11:18:29.169085 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:18:29.169064    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2db6t_69aed169-e8af-429d-99c3-3b20be64e351/ovn-acl-logging/0.log"
Jul 16 11:23:29.188191 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:23:29.188163    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-9lm5s_f64f15a3-5a9f-4737-bc4f-73d5ec311e41/console-operator/1.log"
Jul 16 11:23:29.191806 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:23:29.191784    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-9lm5s_f64f15a3-5a9f-4737-bc4f-73d5ec311e41/console-operator/1.log"
Jul 16 11:23:29.194285 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:23:29.194262    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2db6t_69aed169-e8af-429d-99c3-3b20be64e351/ovn-acl-logging/0.log"
Jul 16 11:23:29.197966 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:23:29.197944    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2db6t_69aed169-e8af-429d-99c3-3b20be64e351/ovn-acl-logging/0.log"
Jul 16 11:28:27.381777 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:27.381740    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-df075e48-openshift-default-58757557b5-fzkpq"]
Jul 16 11:28:27.385623 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:27.385576    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-df075e48-openshift-default-58757557b5-fzkpq"
Jul 16 11:28:27.388051 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:27.388023    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-df075e48-openshift-default-dockercfg-77ml8\""
Jul 16 11:28:27.408223 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:27.408186    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-df075e48-openshift-default-58757557b5-fzkpq"]
Jul 16 11:28:27.484928 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:27.484876    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/09333cd4-4df8-4517-bc05-4e43b79d4740-workload-socket\") pod \"e2e-ait-df075e48-openshift-default-58757557b5-fzkpq\" (UID: \"09333cd4-4df8-4517-bc05-4e43b79d4740\") " pod="openshift-ingress/e2e-ait-df075e48-openshift-default-58757557b5-fzkpq"
Jul 16 11:28:27.484928 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:27.484920    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/09333cd4-4df8-4517-bc05-4e43b79d4740-workload-certs\") pod \"e2e-ait-df075e48-openshift-default-58757557b5-fzkpq\" (UID: \"09333cd4-4df8-4517-bc05-4e43b79d4740\") " pod="openshift-ingress/e2e-ait-df075e48-openshift-default-58757557b5-fzkpq"
Jul 16 11:28:27.485128 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:27.484950    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/09333cd4-4df8-4517-bc05-4e43b79d4740-credential-socket\") pod \"e2e-ait-df075e48-openshift-default-58757557b5-fzkpq\" (UID: \"09333cd4-4df8-4517-bc05-4e43b79d4740\") " pod="openshift-ingress/e2e-ait-df075e48-openshift-default-58757557b5-fzkpq"
Jul 16 11:28:27.485128 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:27.484974    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/09333cd4-4df8-4517-bc05-4e43b79d4740-istio-podinfo\") pod \"e2e-ait-df075e48-openshift-default-58757557b5-fzkpq\" (UID: \"09333cd4-4df8-4517-bc05-4e43b79d4740\") " pod="openshift-ingress/e2e-ait-df075e48-openshift-default-58757557b5-fzkpq"
Jul 16 11:28:27.485128 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:27.485012    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/09333cd4-4df8-4517-bc05-4e43b79d4740-istio-token\") pod \"e2e-ait-df075e48-openshift-default-58757557b5-fzkpq\" (UID: \"09333cd4-4df8-4517-bc05-4e43b79d4740\") " pod="openshift-ingress/e2e-ait-df075e48-openshift-default-58757557b5-fzkpq"
Jul 16 11:28:27.485128 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:27.485076    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/09333cd4-4df8-4517-bc05-4e43b79d4740-istiod-ca-cert\") pod \"e2e-ait-df075e48-openshift-default-58757557b5-fzkpq\" (UID: \"09333cd4-4df8-4517-bc05-4e43b79d4740\") " pod="openshift-ingress/e2e-ait-df075e48-openshift-default-58757557b5-fzkpq"
Jul 16 11:28:27.485256 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:27.485142    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/09333cd4-4df8-4517-bc05-4e43b79d4740-istio-data\") pod \"e2e-ait-df075e48-openshift-default-58757557b5-fzkpq\" (UID: \"09333cd4-4df8-4517-bc05-4e43b79d4740\") " pod="openshift-ingress/e2e-ait-df075e48-openshift-default-58757557b5-fzkpq"
Jul 16 11:28:27.485256 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:27.485206    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/09333cd4-4df8-4517-bc05-4e43b79d4740-istio-envoy\") pod \"e2e-ait-df075e48-openshift-default-58757557b5-fzkpq\" (UID: \"09333cd4-4df8-4517-bc05-4e43b79d4740\") " pod="openshift-ingress/e2e-ait-df075e48-openshift-default-58757557b5-fzkpq"
Jul 16 11:28:27.485256 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:27.485246    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nv9cq\" (UniqueName: \"kubernetes.io/projected/09333cd4-4df8-4517-bc05-4e43b79d4740-kube-api-access-nv9cq\") pod \"e2e-ait-df075e48-openshift-default-58757557b5-fzkpq\" (UID: \"09333cd4-4df8-4517-bc05-4e43b79d4740\") " pod="openshift-ingress/e2e-ait-df075e48-openshift-default-58757557b5-fzkpq"
Jul 16 11:28:27.586310 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:27.586274    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/09333cd4-4df8-4517-bc05-4e43b79d4740-istio-envoy\") pod \"e2e-ait-df075e48-openshift-default-58757557b5-fzkpq\" (UID: \"09333cd4-4df8-4517-bc05-4e43b79d4740\") " pod="openshift-ingress/e2e-ait-df075e48-openshift-default-58757557b5-fzkpq"
Jul 16 11:28:27.586487 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:27.586348    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nv9cq\" (UniqueName: \"kubernetes.io/projected/09333cd4-4df8-4517-bc05-4e43b79d4740-kube-api-access-nv9cq\") pod \"e2e-ait-df075e48-openshift-default-58757557b5-fzkpq\" (UID: \"09333cd4-4df8-4517-bc05-4e43b79d4740\") " pod="openshift-ingress/e2e-ait-df075e48-openshift-default-58757557b5-fzkpq"
Jul 16 11:28:27.586487 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:27.586388    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/09333cd4-4df8-4517-bc05-4e43b79d4740-workload-socket\") pod \"e2e-ait-df075e48-openshift-default-58757557b5-fzkpq\" (UID: \"09333cd4-4df8-4517-bc05-4e43b79d4740\") " pod="openshift-ingress/e2e-ait-df075e48-openshift-default-58757557b5-fzkpq"
Jul 16 11:28:27.586487 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:27.586413    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/09333cd4-4df8-4517-bc05-4e43b79d4740-workload-certs\") pod \"e2e-ait-df075e48-openshift-default-58757557b5-fzkpq\" (UID: \"09333cd4-4df8-4517-bc05-4e43b79d4740\") " pod="openshift-ingress/e2e-ait-df075e48-openshift-default-58757557b5-fzkpq"
Jul 16 11:28:27.586487 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:27.586443    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/09333cd4-4df8-4517-bc05-4e43b79d4740-credential-socket\") pod \"e2e-ait-df075e48-openshift-default-58757557b5-fzkpq\" (UID: \"09333cd4-4df8-4517-bc05-4e43b79d4740\") " pod="openshift-ingress/e2e-ait-df075e48-openshift-default-58757557b5-fzkpq"
Jul 16 11:28:27.586487 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:27.586468    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/09333cd4-4df8-4517-bc05-4e43b79d4740-istio-podinfo\") pod \"e2e-ait-df075e48-openshift-default-58757557b5-fzkpq\" (UID: \"09333cd4-4df8-4517-bc05-4e43b79d4740\") " pod="openshift-ingress/e2e-ait-df075e48-openshift-default-58757557b5-fzkpq"
Jul 16 11:28:27.586807 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:27.586505    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/09333cd4-4df8-4517-bc05-4e43b79d4740-istio-token\") pod \"e2e-ait-df075e48-openshift-default-58757557b5-fzkpq\" (UID: \"09333cd4-4df8-4517-bc05-4e43b79d4740\") " pod="openshift-ingress/e2e-ait-df075e48-openshift-default-58757557b5-fzkpq"
Jul 16 11:28:27.586807 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:27.586545    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/09333cd4-4df8-4517-bc05-4e43b79d4740-istiod-ca-cert\") pod \"e2e-ait-df075e48-openshift-default-58757557b5-fzkpq\" (UID: \"09333cd4-4df8-4517-bc05-4e43b79d4740\") " pod="openshift-ingress/e2e-ait-df075e48-openshift-default-58757557b5-fzkpq"
Jul 16 11:28:27.586807 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:27.586569    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/09333cd4-4df8-4517-bc05-4e43b79d4740-istio-data\") pod \"e2e-ait-df075e48-openshift-default-58757557b5-fzkpq\" (UID: \"09333cd4-4df8-4517-bc05-4e43b79d4740\") " pod="openshift-ingress/e2e-ait-df075e48-openshift-default-58757557b5-fzkpq"
Jul 16 11:28:27.587083 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:27.587055    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/09333cd4-4df8-4517-bc05-4e43b79d4740-workload-certs\") pod \"e2e-ait-df075e48-openshift-default-58757557b5-fzkpq\" (UID: \"09333cd4-4df8-4517-bc05-4e43b79d4740\") " pod="openshift-ingress/e2e-ait-df075e48-openshift-default-58757557b5-fzkpq"
Jul 16 11:28:27.587492 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:27.587463    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/09333cd4-4df8-4517-bc05-4e43b79d4740-workload-socket\") pod \"e2e-ait-df075e48-openshift-default-58757557b5-fzkpq\" (UID: \"09333cd4-4df8-4517-bc05-4e43b79d4740\") " pod="openshift-ingress/e2e-ait-df075e48-openshift-default-58757557b5-fzkpq"
Jul 16 11:28:27.587582 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:27.587489    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/09333cd4-4df8-4517-bc05-4e43b79d4740-credential-socket\") pod \"e2e-ait-df075e48-openshift-default-58757557b5-fzkpq\" (UID: \"09333cd4-4df8-4517-bc05-4e43b79d4740\") " pod="openshift-ingress/e2e-ait-df075e48-openshift-default-58757557b5-fzkpq"
Jul 16 11:28:27.587745 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:27.587720    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/09333cd4-4df8-4517-bc05-4e43b79d4740-istio-data\") pod \"e2e-ait-df075e48-openshift-default-58757557b5-fzkpq\" (UID: \"09333cd4-4df8-4517-bc05-4e43b79d4740\") " pod="openshift-ingress/e2e-ait-df075e48-openshift-default-58757557b5-fzkpq"
Jul 16 11:28:27.587857 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:27.587766    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/09333cd4-4df8-4517-bc05-4e43b79d4740-istiod-ca-cert\") pod \"e2e-ait-df075e48-openshift-default-58757557b5-fzkpq\" (UID: \"09333cd4-4df8-4517-bc05-4e43b79d4740\") " pod="openshift-ingress/e2e-ait-df075e48-openshift-default-58757557b5-fzkpq"
Jul 16 11:28:27.589145 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:27.589120    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/09333cd4-4df8-4517-bc05-4e43b79d4740-istio-envoy\") pod \"e2e-ait-df075e48-openshift-default-58757557b5-fzkpq\" (UID: \"09333cd4-4df8-4517-bc05-4e43b79d4740\") " pod="openshift-ingress/e2e-ait-df075e48-openshift-default-58757557b5-fzkpq"
Jul 16 11:28:27.590105 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:27.590079    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/09333cd4-4df8-4517-bc05-4e43b79d4740-istio-podinfo\") pod \"e2e-ait-df075e48-openshift-default-58757557b5-fzkpq\" (UID: \"09333cd4-4df8-4517-bc05-4e43b79d4740\") " pod="openshift-ingress/e2e-ait-df075e48-openshift-default-58757557b5-fzkpq"
Jul 16 11:28:27.604830 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:27.604793    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-df075e48-8958f6c87-5fmw6"]
Jul 16 11:28:27.609124 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:27.609089    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-df075e48-8958f6c87-5fmw6"
Jul 16 11:28:27.609866 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:27.609840    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nv9cq\" (UniqueName: \"kubernetes.io/projected/09333cd4-4df8-4517-bc05-4e43b79d4740-kube-api-access-nv9cq\") pod \"e2e-ait-df075e48-openshift-default-58757557b5-fzkpq\" (UID: \"09333cd4-4df8-4517-bc05-4e43b79d4740\") " pod="openshift-ingress/e2e-ait-df075e48-openshift-default-58757557b5-fzkpq"
Jul 16 11:28:27.610008 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:27.609989    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/09333cd4-4df8-4517-bc05-4e43b79d4740-istio-token\") pod \"e2e-ait-df075e48-openshift-default-58757557b5-fzkpq\" (UID: \"09333cd4-4df8-4517-bc05-4e43b79d4740\") " pod="openshift-ingress/e2e-ait-df075e48-openshift-default-58757557b5-fzkpq"
Jul 16 11:28:27.611564 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:27.611545    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-df075e48-serving-cert\""
Jul 16 11:28:27.618990 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:27.618959    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-df075e48-8958f6c87-5fmw6"]
Jul 16 11:28:27.697676 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:27.697640    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-df075e48-openshift-default-58757557b5-fzkpq"
Jul 16 11:28:27.788459 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:27.788428    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7e7bb44b-d665-49c8-b594-f203b73508bc-maas-api-tls\") pod \"maas-api-e2e-ait-df075e48-8958f6c87-5fmw6\" (UID: \"7e7bb44b-d665-49c8-b594-f203b73508bc\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-df075e48-8958f6c87-5fmw6"
Jul 16 11:28:27.788459 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:27.788461    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7e7bb44b-d665-49c8-b594-f203b73508bc-openshift-service-ca\") pod \"maas-api-e2e-ait-df075e48-8958f6c87-5fmw6\" (UID: \"7e7bb44b-d665-49c8-b594-f203b73508bc\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-df075e48-8958f6c87-5fmw6"
Jul 16 11:28:27.788752 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:27.788635    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wrscv\" (UniqueName: \"kubernetes.io/projected/7e7bb44b-d665-49c8-b594-f203b73508bc-kube-api-access-wrscv\") pod \"maas-api-e2e-ait-df075e48-8958f6c87-5fmw6\" (UID: \"7e7bb44b-d665-49c8-b594-f203b73508bc\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-df075e48-8958f6c87-5fmw6"
Jul 16 11:28:27.863498 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:27.863470    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-df075e48-openshift-default-58757557b5-fzkpq"]
Jul 16 11:28:27.865420 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:28:27.865395    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09333cd4_4df8_4517_bc05_4e43b79d4740.slice/crio-d1d6728ebcd60c1575ac978d447a14f7002c94671d6840eb038df361bbfccbfb WatchSource:0}: Error finding container d1d6728ebcd60c1575ac978d447a14f7002c94671d6840eb038df361bbfccbfb: Status 404 returned error can't find the container with id d1d6728ebcd60c1575ac978d447a14f7002c94671d6840eb038df361bbfccbfb
Jul 16 11:28:27.867441 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:27.867410    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 11:28:27.867522 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:27.867481    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 11:28:27.867522 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:27.867514    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 11:28:27.890083 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:27.890057    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wrscv\" (UniqueName: \"kubernetes.io/projected/7e7bb44b-d665-49c8-b594-f203b73508bc-kube-api-access-wrscv\") pod \"maas-api-e2e-ait-df075e48-8958f6c87-5fmw6\" (UID: \"7e7bb44b-d665-49c8-b594-f203b73508bc\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-df075e48-8958f6c87-5fmw6"
Jul 16 11:28:27.890216 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:27.890095    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7e7bb44b-d665-49c8-b594-f203b73508bc-maas-api-tls\") pod \"maas-api-e2e-ait-df075e48-8958f6c87-5fmw6\" (UID: \"7e7bb44b-d665-49c8-b594-f203b73508bc\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-df075e48-8958f6c87-5fmw6"
Jul 16 11:28:27.890216 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:27.890116    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7e7bb44b-d665-49c8-b594-f203b73508bc-openshift-service-ca\") pod \"maas-api-e2e-ait-df075e48-8958f6c87-5fmw6\" (UID: \"7e7bb44b-d665-49c8-b594-f203b73508bc\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-df075e48-8958f6c87-5fmw6"
Jul 16 11:28:27.890936 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:27.890888    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7e7bb44b-d665-49c8-b594-f203b73508bc-openshift-service-ca\") pod \"maas-api-e2e-ait-df075e48-8958f6c87-5fmw6\" (UID: \"7e7bb44b-d665-49c8-b594-f203b73508bc\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-df075e48-8958f6c87-5fmw6"
Jul 16 11:28:27.892786 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:27.892766    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7e7bb44b-d665-49c8-b594-f203b73508bc-maas-api-tls\") pod \"maas-api-e2e-ait-df075e48-8958f6c87-5fmw6\" (UID: \"7e7bb44b-d665-49c8-b594-f203b73508bc\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-df075e48-8958f6c87-5fmw6"
Jul 16 11:28:27.901142 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:27.901115    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wrscv\" (UniqueName: \"kubernetes.io/projected/7e7bb44b-d665-49c8-b594-f203b73508bc-kube-api-access-wrscv\") pod \"maas-api-e2e-ait-df075e48-8958f6c87-5fmw6\" (UID: \"7e7bb44b-d665-49c8-b594-f203b73508bc\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-df075e48-8958f6c87-5fmw6"
Jul 16 11:28:27.935960 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:27.935929    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-df075e48-8958f6c87-5fmw6"
Jul 16 11:28:28.094324 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:28.094295    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-df075e48-8958f6c87-5fmw6"]
Jul 16 11:28:28.096513 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:28:28.096485    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e7bb44b_d665_49c8_b594_f203b73508bc.slice/crio-e727a7249c804dbe642224004f309ce0acd651a2c27500643dd4f9b9c53e755b WatchSource:0}: Error finding container e727a7249c804dbe642224004f309ce0acd651a2c27500643dd4f9b9c53e755b: Status 404 returned error can't find the container with id e727a7249c804dbe642224004f309ce0acd651a2c27500643dd4f9b9c53e755b
Jul 16 11:28:28.097939 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:28.097922    2583 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 16 11:28:28.448904 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:28.448861    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-df075e48-8958f6c87-5fmw6" event={"ID":"7e7bb44b-d665-49c8-b594-f203b73508bc","Type":"ContainerStarted","Data":"e727a7249c804dbe642224004f309ce0acd651a2c27500643dd4f9b9c53e755b"}
Jul 16 11:28:28.450680 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:28.450640    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-df075e48-openshift-default-58757557b5-fzkpq" event={"ID":"09333cd4-4df8-4517-bc05-4e43b79d4740","Type":"ContainerStarted","Data":"99b2c88950c27a36466e830fcf64a38b5046d1bf3996d51b66a944cbb7644f96"}
Jul 16 11:28:28.450825 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:28.450699    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-df075e48-openshift-default-58757557b5-fzkpq" event={"ID":"09333cd4-4df8-4517-bc05-4e43b79d4740","Type":"ContainerStarted","Data":"d1d6728ebcd60c1575ac978d447a14f7002c94671d6840eb038df361bbfccbfb"}
Jul 16 11:28:28.478396 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:28.478344    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-df075e48-openshift-default-58757557b5-fzkpq" podStartSLOduration=1.478328917 podStartE2EDuration="1.478328917s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 11:28:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 11:28:28.476076104 +0000 UTC m=+1501.110297563" watchObservedRunningTime="2026-07-16 11:28:28.478328917 +0000 UTC m=+1501.112550371"
Jul 16 11:28:28.698448 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:28.698414    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-df075e48-openshift-default-58757557b5-fzkpq"
Jul 16 11:28:28.703962 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:28.703881    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-df075e48-openshift-default-58757557b5-fzkpq"
Jul 16 11:28:29.227349 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:29.227316    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-9lm5s_f64f15a3-5a9f-4737-bc4f-73d5ec311e41/console-operator/1.log"
Jul 16 11:28:29.229832 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:29.229807    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-9lm5s_f64f15a3-5a9f-4737-bc4f-73d5ec311e41/console-operator/1.log"
Jul 16 11:28:29.234050 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:29.234025    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2db6t_69aed169-e8af-429d-99c3-3b20be64e351/ovn-acl-logging/0.log"
Jul 16 11:28:29.235962 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:29.235944    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2db6t_69aed169-e8af-429d-99c3-3b20be64e351/ovn-acl-logging/0.log"
Jul 16 11:28:29.457039 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:29.457000    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-df075e48-8958f6c87-5fmw6" event={"ID":"7e7bb44b-d665-49c8-b594-f203b73508bc","Type":"ContainerStarted","Data":"bb2ccfb018f59022fc03c296ac0afa401e77caaf1b6c848267caf3e5825ab442"}
Jul 16 11:28:29.457444 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:29.457321    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-df075e48-openshift-default-58757557b5-fzkpq"
Jul 16 11:28:29.458198 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:29.458177    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-df075e48-openshift-default-58757557b5-fzkpq"
Jul 16 11:28:29.474590 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:29.474532    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-df075e48-8958f6c87-5fmw6" podStartSLOduration=1.7214294140000002 podStartE2EDuration="2.47451684s" totalImagesPullingTime="753.087426ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 11:28:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 11:28:29.473490927 +0000 UTC m=+1502.107712383" watchObservedRunningTime="2026-07-16 11:28:29.47451684 +0000 UTC m=+1502.108738293"
Jul 16 11:28:30.462100 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:30.462069    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-df075e48-8958f6c87-5fmw6"
Jul 16 11:28:33.781529 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:33.781492    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-df075e48-openshift-default-58757557b5-fzkpq"]
Jul 16 11:28:33.781940 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:33.781728    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-df075e48-openshift-default-58757557b5-fzkpq" podUID="09333cd4-4df8-4517-bc05-4e43b79d4740" containerName="istio-proxy" containerID="cri-o://99b2c88950c27a36466e830fcf64a38b5046d1bf3996d51b66a944cbb7644f96" gracePeriod=30
Jul 16 11:28:36.472329 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:36.472300    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-df075e48-8958f6c87-5fmw6"
Jul 16 11:28:40.244140 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:40.244105    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-df075e48-openshift-default-58757557b5-fzkpq"
Jul 16 11:28:40.307463 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:40.307375    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-df075e48-8958f6c87-5fmw6"]
Jul 16 11:28:40.307664 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:40.307633    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-df075e48-8958f6c87-5fmw6" podUID="7e7bb44b-d665-49c8-b594-f203b73508bc" containerName="maas-api" containerID="cri-o://bb2ccfb018f59022fc03c296ac0afa401e77caaf1b6c848267caf3e5825ab442" gracePeriod=30
Jul 16 11:28:40.313982 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:40.313836    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/09333cd4-4df8-4517-bc05-4e43b79d4740-istio-token\") pod \"09333cd4-4df8-4517-bc05-4e43b79d4740\" (UID: \"09333cd4-4df8-4517-bc05-4e43b79d4740\") "
Jul 16 11:28:40.313982 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:40.313893    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/09333cd4-4df8-4517-bc05-4e43b79d4740-workload-socket\") pod \"09333cd4-4df8-4517-bc05-4e43b79d4740\" (UID: \"09333cd4-4df8-4517-bc05-4e43b79d4740\") "
Jul 16 11:28:40.313982 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:40.313926    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/09333cd4-4df8-4517-bc05-4e43b79d4740-istio-envoy\") pod \"09333cd4-4df8-4517-bc05-4e43b79d4740\" (UID: \"09333cd4-4df8-4517-bc05-4e43b79d4740\") "
Jul 16 11:28:40.314212 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:40.313997    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/09333cd4-4df8-4517-bc05-4e43b79d4740-credential-socket\") pod \"09333cd4-4df8-4517-bc05-4e43b79d4740\" (UID: \"09333cd4-4df8-4517-bc05-4e43b79d4740\") "
Jul 16 11:28:40.314212 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:40.314036    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/09333cd4-4df8-4517-bc05-4e43b79d4740-istio-podinfo\") pod \"09333cd4-4df8-4517-bc05-4e43b79d4740\" (UID: \"09333cd4-4df8-4517-bc05-4e43b79d4740\") "
Jul 16 11:28:40.314212 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:40.314059    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nv9cq\" (UniqueName: \"kubernetes.io/projected/09333cd4-4df8-4517-bc05-4e43b79d4740-kube-api-access-nv9cq\") pod \"09333cd4-4df8-4517-bc05-4e43b79d4740\" (UID: \"09333cd4-4df8-4517-bc05-4e43b79d4740\") "
Jul 16 11:28:40.314212 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:40.314094    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/09333cd4-4df8-4517-bc05-4e43b79d4740-istio-data\") pod \"09333cd4-4df8-4517-bc05-4e43b79d4740\" (UID: \"09333cd4-4df8-4517-bc05-4e43b79d4740\") "
Jul 16 11:28:40.314212 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:40.314133    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/09333cd4-4df8-4517-bc05-4e43b79d4740-workload-certs\") pod \"09333cd4-4df8-4517-bc05-4e43b79d4740\" (UID: \"09333cd4-4df8-4517-bc05-4e43b79d4740\") "
Jul 16 11:28:40.314212 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:40.314178    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/09333cd4-4df8-4517-bc05-4e43b79d4740-istiod-ca-cert\") pod \"09333cd4-4df8-4517-bc05-4e43b79d4740\" (UID: \"09333cd4-4df8-4517-bc05-4e43b79d4740\") "
Jul 16 11:28:40.314738 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:40.314418    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/09333cd4-4df8-4517-bc05-4e43b79d4740-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "09333cd4-4df8-4517-bc05-4e43b79d4740" (UID: "09333cd4-4df8-4517-bc05-4e43b79d4740"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 11:28:40.314738 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:40.314517    2583 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/09333cd4-4df8-4517-bc05-4e43b79d4740-credential-socket\") on node \"ip-10-0-129-28.ec2.internal\" DevicePath \"\""
Jul 16 11:28:40.315461 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:40.315429    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/09333cd4-4df8-4517-bc05-4e43b79d4740-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "09333cd4-4df8-4517-bc05-4e43b79d4740" (UID: "09333cd4-4df8-4517-bc05-4e43b79d4740"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 11:28:40.315550 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:40.315506    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/09333cd4-4df8-4517-bc05-4e43b79d4740-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "09333cd4-4df8-4517-bc05-4e43b79d4740" (UID: "09333cd4-4df8-4517-bc05-4e43b79d4740"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 11:28:40.315625 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:40.315591    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/09333cd4-4df8-4517-bc05-4e43b79d4740-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "09333cd4-4df8-4517-bc05-4e43b79d4740" (UID: "09333cd4-4df8-4517-bc05-4e43b79d4740"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 11:28:40.316479 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:40.315854    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/09333cd4-4df8-4517-bc05-4e43b79d4740-istio-data" (OuterVolumeSpecName: "istio-data") pod "09333cd4-4df8-4517-bc05-4e43b79d4740" (UID: "09333cd4-4df8-4517-bc05-4e43b79d4740"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 11:28:40.316914 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:40.316883    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/09333cd4-4df8-4517-bc05-4e43b79d4740-istio-token" (OuterVolumeSpecName: "istio-token") pod "09333cd4-4df8-4517-bc05-4e43b79d4740" (UID: "09333cd4-4df8-4517-bc05-4e43b79d4740"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 11:28:40.317895 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:40.317867    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/09333cd4-4df8-4517-bc05-4e43b79d4740-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "09333cd4-4df8-4517-bc05-4e43b79d4740" (UID: "09333cd4-4df8-4517-bc05-4e43b79d4740"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 16 11:28:40.318002 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:40.317979    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/09333cd4-4df8-4517-bc05-4e43b79d4740-kube-api-access-nv9cq" (OuterVolumeSpecName: "kube-api-access-nv9cq") pod "09333cd4-4df8-4517-bc05-4e43b79d4740" (UID: "09333cd4-4df8-4517-bc05-4e43b79d4740"). InnerVolumeSpecName "kube-api-access-nv9cq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 11:28:40.318410 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:40.318379    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/09333cd4-4df8-4517-bc05-4e43b79d4740-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "09333cd4-4df8-4517-bc05-4e43b79d4740" (UID: "09333cd4-4df8-4517-bc05-4e43b79d4740"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 11:28:40.415082 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:40.415054    2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/09333cd4-4df8-4517-bc05-4e43b79d4740-istio-token\") on node \"ip-10-0-129-28.ec2.internal\" DevicePath \"\""
Jul 16 11:28:40.415082 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:40.415080    2583 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/09333cd4-4df8-4517-bc05-4e43b79d4740-workload-socket\") on node \"ip-10-0-129-28.ec2.internal\" DevicePath \"\""
Jul 16 11:28:40.415082 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:40.415091    2583 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/09333cd4-4df8-4517-bc05-4e43b79d4740-istio-envoy\") on node \"ip-10-0-129-28.ec2.internal\" DevicePath \"\""
Jul 16 11:28:40.415304 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:40.415100    2583 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/09333cd4-4df8-4517-bc05-4e43b79d4740-istio-podinfo\") on node \"ip-10-0-129-28.ec2.internal\" DevicePath \"\""
Jul 16 11:28:40.415304 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:40.415109    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nv9cq\" (UniqueName: \"kubernetes.io/projected/09333cd4-4df8-4517-bc05-4e43b79d4740-kube-api-access-nv9cq\") on node \"ip-10-0-129-28.ec2.internal\" DevicePath \"\""
Jul 16 11:28:40.415304 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:40.415117    2583 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/09333cd4-4df8-4517-bc05-4e43b79d4740-istio-data\") on node \"ip-10-0-129-28.ec2.internal\" DevicePath \"\""
Jul 16 11:28:40.415304 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:40.415126    2583 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/09333cd4-4df8-4517-bc05-4e43b79d4740-workload-certs\") on node \"ip-10-0-129-28.ec2.internal\" DevicePath \"\""
Jul 16 11:28:40.415304 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:40.415135    2583 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/09333cd4-4df8-4517-bc05-4e43b79d4740-istiod-ca-cert\") on node \"ip-10-0-129-28.ec2.internal\" DevicePath \"\""
Jul 16 11:28:40.511789 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:40.511747    2583 generic.go:356] "Generic (PLEG): container finished" podID="09333cd4-4df8-4517-bc05-4e43b79d4740" containerID="99b2c88950c27a36466e830fcf64a38b5046d1bf3996d51b66a944cbb7644f96" exitCode=0
Jul 16 11:28:40.511966 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:40.511802    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-df075e48-openshift-default-58757557b5-fzkpq"
Jul 16 11:28:40.511966 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:40.511936    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-df075e48-openshift-default-58757557b5-fzkpq" event={"ID":"09333cd4-4df8-4517-bc05-4e43b79d4740","Type":"ContainerDied","Data":"99b2c88950c27a36466e830fcf64a38b5046d1bf3996d51b66a944cbb7644f96"}
Jul 16 11:28:40.512070 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:40.511985    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-df075e48-openshift-default-58757557b5-fzkpq" event={"ID":"09333cd4-4df8-4517-bc05-4e43b79d4740","Type":"ContainerDied","Data":"d1d6728ebcd60c1575ac978d447a14f7002c94671d6840eb038df361bbfccbfb"}
Jul 16 11:28:40.512070 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:40.512013    2583 scope.go:117] "RemoveContainer" containerID="99b2c88950c27a36466e830fcf64a38b5046d1bf3996d51b66a944cbb7644f96"
Jul 16 11:28:40.513637 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:40.513583    2583 generic.go:356] "Generic (PLEG): container finished" podID="7e7bb44b-d665-49c8-b594-f203b73508bc" containerID="bb2ccfb018f59022fc03c296ac0afa401e77caaf1b6c848267caf3e5825ab442" exitCode=0
Jul 16 11:28:40.513756 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:40.513632    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-df075e48-8958f6c87-5fmw6" event={"ID":"7e7bb44b-d665-49c8-b594-f203b73508bc","Type":"ContainerDied","Data":"bb2ccfb018f59022fc03c296ac0afa401e77caaf1b6c848267caf3e5825ab442"}
Jul 16 11:28:40.521945 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:40.521925    2583 scope.go:117] "RemoveContainer" containerID="99b2c88950c27a36466e830fcf64a38b5046d1bf3996d51b66a944cbb7644f96"
Jul 16 11:28:40.522264 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:28:40.522238    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"99b2c88950c27a36466e830fcf64a38b5046d1bf3996d51b66a944cbb7644f96\": container with ID starting with 99b2c88950c27a36466e830fcf64a38b5046d1bf3996d51b66a944cbb7644f96 not found: ID does not exist" containerID="99b2c88950c27a36466e830fcf64a38b5046d1bf3996d51b66a944cbb7644f96"
Jul 16 11:28:40.522353 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:40.522278    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"99b2c88950c27a36466e830fcf64a38b5046d1bf3996d51b66a944cbb7644f96"} err="failed to get container status \"99b2c88950c27a36466e830fcf64a38b5046d1bf3996d51b66a944cbb7644f96\": rpc error: code = NotFound desc = could not find container \"99b2c88950c27a36466e830fcf64a38b5046d1bf3996d51b66a944cbb7644f96\": container with ID starting with 99b2c88950c27a36466e830fcf64a38b5046d1bf3996d51b66a944cbb7644f96 not found: ID does not exist"
Jul 16 11:28:40.536214 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:40.536189    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-df075e48-openshift-default-58757557b5-fzkpq"]
Jul 16 11:28:40.544027 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:40.544003    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-df075e48-openshift-default-58757557b5-fzkpq"]
Jul 16 11:28:40.547673 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:40.547653    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-df075e48-8958f6c87-5fmw6"
Jul 16 11:28:40.616723 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:40.616619    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7e7bb44b-d665-49c8-b594-f203b73508bc-openshift-service-ca\") pod \"7e7bb44b-d665-49c8-b594-f203b73508bc\" (UID: \"7e7bb44b-d665-49c8-b594-f203b73508bc\") "
Jul 16 11:28:40.616723 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:40.616682    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7e7bb44b-d665-49c8-b594-f203b73508bc-maas-api-tls\") pod \"7e7bb44b-d665-49c8-b594-f203b73508bc\" (UID: \"7e7bb44b-d665-49c8-b594-f203b73508bc\") "
Jul 16 11:28:40.616954 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:40.616764    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wrscv\" (UniqueName: \"kubernetes.io/projected/7e7bb44b-d665-49c8-b594-f203b73508bc-kube-api-access-wrscv\") pod \"7e7bb44b-d665-49c8-b594-f203b73508bc\" (UID: \"7e7bb44b-d665-49c8-b594-f203b73508bc\") "
Jul 16 11:28:40.617007 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:40.616971    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7e7bb44b-d665-49c8-b594-f203b73508bc-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "7e7bb44b-d665-49c8-b594-f203b73508bc" (UID: "7e7bb44b-d665-49c8-b594-f203b73508bc"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 11:28:40.617103 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:40.617087    2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7e7bb44b-d665-49c8-b594-f203b73508bc-openshift-service-ca\") on node \"ip-10-0-129-28.ec2.internal\" DevicePath \"\""
Jul 16 11:28:40.618983 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:40.618959    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7e7bb44b-d665-49c8-b594-f203b73508bc-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "7e7bb44b-d665-49c8-b594-f203b73508bc" (UID: "7e7bb44b-d665-49c8-b594-f203b73508bc"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 11:28:40.618983 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:40.618973    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7e7bb44b-d665-49c8-b594-f203b73508bc-kube-api-access-wrscv" (OuterVolumeSpecName: "kube-api-access-wrscv") pod "7e7bb44b-d665-49c8-b594-f203b73508bc" (UID: "7e7bb44b-d665-49c8-b594-f203b73508bc"). InnerVolumeSpecName "kube-api-access-wrscv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 11:28:40.718238 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:40.718203    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7e7bb44b-d665-49c8-b594-f203b73508bc-maas-api-tls\") on node \"ip-10-0-129-28.ec2.internal\" DevicePath \"\""
Jul 16 11:28:40.718238 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:40.718235    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wrscv\" (UniqueName: \"kubernetes.io/projected/7e7bb44b-d665-49c8-b594-f203b73508bc-kube-api-access-wrscv\") on node \"ip-10-0-129-28.ec2.internal\" DevicePath \"\""
Jul 16 11:28:41.228717 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:41.228679    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="09333cd4-4df8-4517-bc05-4e43b79d4740" path="/var/lib/kubelet/pods/09333cd4-4df8-4517-bc05-4e43b79d4740/volumes"
Jul 16 11:28:41.520026 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:41.519995    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-df075e48-8958f6c87-5fmw6"
Jul 16 11:28:41.520452 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:41.520004    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-df075e48-8958f6c87-5fmw6" event={"ID":"7e7bb44b-d665-49c8-b594-f203b73508bc","Type":"ContainerDied","Data":"e727a7249c804dbe642224004f309ce0acd651a2c27500643dd4f9b9c53e755b"}
Jul 16 11:28:41.520452 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:41.520132    2583 scope.go:117] "RemoveContainer" containerID="bb2ccfb018f59022fc03c296ac0afa401e77caaf1b6c848267caf3e5825ab442"
Jul 16 11:28:41.541557 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:41.541523    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-df075e48-8958f6c87-5fmw6"]
Jul 16 11:28:41.545783 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:41.545754    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-df075e48-8958f6c87-5fmw6"]
Jul 16 11:28:43.228305 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:43.228267    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7e7bb44b-d665-49c8-b594-f203b73508bc" path="/var/lib/kubelet/pods/7e7bb44b-d665-49c8-b594-f203b73508bc/volumes"
Jul 16 11:28:46.916756 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:46.916725    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-d6ad08cc-gw-openshift-default-787fcd9587-bs976"]
Jul 16 11:28:46.917097 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:46.917083    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7e7bb44b-d665-49c8-b594-f203b73508bc" containerName="maas-api"
Jul 16 11:28:46.917146 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:46.917098    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="7e7bb44b-d665-49c8-b594-f203b73508bc" containerName="maas-api"
Jul 16 11:28:46.917146 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:46.917111    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="09333cd4-4df8-4517-bc05-4e43b79d4740" containerName="istio-proxy"
Jul 16 11:28:46.917146 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:46.917117    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="09333cd4-4df8-4517-bc05-4e43b79d4740" containerName="istio-proxy"
Jul 16 11:28:46.917238 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:46.917178    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="09333cd4-4df8-4517-bc05-4e43b79d4740" containerName="istio-proxy"
Jul 16 11:28:46.917238 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:46.917188    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="7e7bb44b-d665-49c8-b594-f203b73508bc" containerName="maas-api"
Jul 16 11:28:46.921423 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:46.921404    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-d6ad08cc-gw-openshift-default-787fcd9587-bs976"
Jul 16 11:28:46.925846 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:46.925825    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-d6ad08cc-gw-openshift-default-dockercfg-zcdgg\""
Jul 16 11:28:46.940996 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:46.940963    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-d6ad08cc-gw-openshift-default-787fcd9587-bs976"]
Jul 16 11:28:46.971560 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:46.971516    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f4b8f388-11c3-4acb-a008-501c23839953-istiod-ca-cert\") pod \"e2e-migrate-d6ad08cc-gw-openshift-default-787fcd9587-bs976\" (UID: \"f4b8f388-11c3-4acb-a008-501c23839953\") " pod="openshift-ingress/e2e-migrate-d6ad08cc-gw-openshift-default-787fcd9587-bs976"
Jul 16 11:28:46.971560 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:46.971563    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4b8f388-11c3-4acb-a008-501c23839953-workload-socket\") pod \"e2e-migrate-d6ad08cc-gw-openshift-default-787fcd9587-bs976\" (UID: \"f4b8f388-11c3-4acb-a008-501c23839953\") " pod="openshift-ingress/e2e-migrate-d6ad08cc-gw-openshift-default-787fcd9587-bs976"
Jul 16 11:28:46.971835 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:46.971587    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4b8f388-11c3-4acb-a008-501c23839953-credential-socket\") pod \"e2e-migrate-d6ad08cc-gw-openshift-default-787fcd9587-bs976\" (UID: \"f4b8f388-11c3-4acb-a008-501c23839953\") " pod="openshift-ingress/e2e-migrate-d6ad08cc-gw-openshift-default-787fcd9587-bs976"
Jul 16 11:28:46.971835 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:46.971626    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f4b8f388-11c3-4acb-a008-501c23839953-istio-envoy\") pod \"e2e-migrate-d6ad08cc-gw-openshift-default-787fcd9587-bs976\" (UID: \"f4b8f388-11c3-4acb-a008-501c23839953\") " pod="openshift-ingress/e2e-migrate-d6ad08cc-gw-openshift-default-787fcd9587-bs976"
Jul 16 11:28:46.971835 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:46.971651    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f4b8f388-11c3-4acb-a008-501c23839953-istio-podinfo\") pod \"e2e-migrate-d6ad08cc-gw-openshift-default-787fcd9587-bs976\" (UID: \"f4b8f388-11c3-4acb-a008-501c23839953\") " pod="openshift-ingress/e2e-migrate-d6ad08cc-gw-openshift-default-787fcd9587-bs976"
Jul 16 11:28:46.971835 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:46.971684    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f4b8f388-11c3-4acb-a008-501c23839953-workload-certs\") pod \"e2e-migrate-d6ad08cc-gw-openshift-default-787fcd9587-bs976\" (UID: \"f4b8f388-11c3-4acb-a008-501c23839953\") " pod="openshift-ingress/e2e-migrate-d6ad08cc-gw-openshift-default-787fcd9587-bs976"
Jul 16 11:28:46.971835 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:46.971706    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f4b8f388-11c3-4acb-a008-501c23839953-istio-token\") pod \"e2e-migrate-d6ad08cc-gw-openshift-default-787fcd9587-bs976\" (UID: \"f4b8f388-11c3-4acb-a008-501c23839953\") " pod="openshift-ingress/e2e-migrate-d6ad08cc-gw-openshift-default-787fcd9587-bs976"
Jul 16 11:28:46.971835 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:46.971729    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qkmnz\" (UniqueName: \"kubernetes.io/projected/f4b8f388-11c3-4acb-a008-501c23839953-kube-api-access-qkmnz\") pod \"e2e-migrate-d6ad08cc-gw-openshift-default-787fcd9587-bs976\" (UID: \"f4b8f388-11c3-4acb-a008-501c23839953\") " pod="openshift-ingress/e2e-migrate-d6ad08cc-gw-openshift-default-787fcd9587-bs976"
Jul 16 11:28:46.972044 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:46.971848    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f4b8f388-11c3-4acb-a008-501c23839953-istio-data\") pod \"e2e-migrate-d6ad08cc-gw-openshift-default-787fcd9587-bs976\" (UID: \"f4b8f388-11c3-4acb-a008-501c23839953\") " pod="openshift-ingress/e2e-migrate-d6ad08cc-gw-openshift-default-787fcd9587-bs976"
Jul 16 11:28:47.072582 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:47.072532    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f4b8f388-11c3-4acb-a008-501c23839953-istiod-ca-cert\") pod \"e2e-migrate-d6ad08cc-gw-openshift-default-787fcd9587-bs976\" (UID: \"f4b8f388-11c3-4acb-a008-501c23839953\") " pod="openshift-ingress/e2e-migrate-d6ad08cc-gw-openshift-default-787fcd9587-bs976"
Jul 16 11:28:47.072582 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:47.072583    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4b8f388-11c3-4acb-a008-501c23839953-workload-socket\") pod \"e2e-migrate-d6ad08cc-gw-openshift-default-787fcd9587-bs976\" (UID: \"f4b8f388-11c3-4acb-a008-501c23839953\") " pod="openshift-ingress/e2e-migrate-d6ad08cc-gw-openshift-default-787fcd9587-bs976"
Jul 16 11:28:47.072839 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:47.072636    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4b8f388-11c3-4acb-a008-501c23839953-credential-socket\") pod \"e2e-migrate-d6ad08cc-gw-openshift-default-787fcd9587-bs976\" (UID: \"f4b8f388-11c3-4acb-a008-501c23839953\") " pod="openshift-ingress/e2e-migrate-d6ad08cc-gw-openshift-default-787fcd9587-bs976"
Jul 16 11:28:47.072839 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:47.072660    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f4b8f388-11c3-4acb-a008-501c23839953-istio-envoy\") pod \"e2e-migrate-d6ad08cc-gw-openshift-default-787fcd9587-bs976\" (UID: \"f4b8f388-11c3-4acb-a008-501c23839953\") " pod="openshift-ingress/e2e-migrate-d6ad08cc-gw-openshift-default-787fcd9587-bs976"
Jul 16 11:28:47.072839 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:47.072690    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f4b8f388-11c3-4acb-a008-501c23839953-istio-podinfo\") pod \"e2e-migrate-d6ad08cc-gw-openshift-default-787fcd9587-bs976\" (UID: \"f4b8f388-11c3-4acb-a008-501c23839953\") " pod="openshift-ingress/e2e-migrate-d6ad08cc-gw-openshift-default-787fcd9587-bs976"
Jul 16 11:28:47.072839 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:47.072732    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f4b8f388-11c3-4acb-a008-501c23839953-workload-certs\") pod \"e2e-migrate-d6ad08cc-gw-openshift-default-787fcd9587-bs976\" (UID: \"f4b8f388-11c3-4acb-a008-501c23839953\") " pod="openshift-ingress/e2e-migrate-d6ad08cc-gw-openshift-default-787fcd9587-bs976"
Jul 16 11:28:47.073071 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:47.072890    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f4b8f388-11c3-4acb-a008-501c23839953-istio-token\") pod \"e2e-migrate-d6ad08cc-gw-openshift-default-787fcd9587-bs976\" (UID: \"f4b8f388-11c3-4acb-a008-501c23839953\") " pod="openshift-ingress/e2e-migrate-d6ad08cc-gw-openshift-default-787fcd9587-bs976"
Jul 16 11:28:47.073071 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:47.072941    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qkmnz\" (UniqueName: \"kubernetes.io/projected/f4b8f388-11c3-4acb-a008-501c23839953-kube-api-access-qkmnz\") pod \"e2e-migrate-d6ad08cc-gw-openshift-default-787fcd9587-bs976\" (UID: \"f4b8f388-11c3-4acb-a008-501c23839953\") " pod="openshift-ingress/e2e-migrate-d6ad08cc-gw-openshift-default-787fcd9587-bs976"
Jul 16 11:28:47.073071 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:47.073040    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f4b8f388-11c3-4acb-a008-501c23839953-istio-data\") pod \"e2e-migrate-d6ad08cc-gw-openshift-default-787fcd9587-bs976\" (UID: \"f4b8f388-11c3-4acb-a008-501c23839953\") " pod="openshift-ingress/e2e-migrate-d6ad08cc-gw-openshift-default-787fcd9587-bs976"
Jul 16 11:28:47.073269 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:47.073120    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4b8f388-11c3-4acb-a008-501c23839953-credential-socket\") pod \"e2e-migrate-d6ad08cc-gw-openshift-default-787fcd9587-bs976\" (UID: \"f4b8f388-11c3-4acb-a008-501c23839953\") " pod="openshift-ingress/e2e-migrate-d6ad08cc-gw-openshift-default-787fcd9587-bs976"
Jul 16 11:28:47.073269 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:47.073217    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f4b8f388-11c3-4acb-a008-501c23839953-workload-certs\") pod \"e2e-migrate-d6ad08cc-gw-openshift-default-787fcd9587-bs976\" (UID: \"f4b8f388-11c3-4acb-a008-501c23839953\") " pod="openshift-ingress/e2e-migrate-d6ad08cc-gw-openshift-default-787fcd9587-bs976"
Jul 16 11:28:47.073366 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:47.073326    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4b8f388-11c3-4acb-a008-501c23839953-workload-socket\") pod \"e2e-migrate-d6ad08cc-gw-openshift-default-787fcd9587-bs976\" (UID: \"f4b8f388-11c3-4acb-a008-501c23839953\") " pod="openshift-ingress/e2e-migrate-d6ad08cc-gw-openshift-default-787fcd9587-bs976"
Jul 16 11:28:47.073427 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:47.073384    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f4b8f388-11c3-4acb-a008-501c23839953-istio-data\") pod \"e2e-migrate-d6ad08cc-gw-openshift-default-787fcd9587-bs976\" (UID: \"f4b8f388-11c3-4acb-a008-501c23839953\") " pod="openshift-ingress/e2e-migrate-d6ad08cc-gw-openshift-default-787fcd9587-bs976"
Jul 16 11:28:47.073473 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:47.073457    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f4b8f388-11c3-4acb-a008-501c23839953-istiod-ca-cert\") pod \"e2e-migrate-d6ad08cc-gw-openshift-default-787fcd9587-bs976\" (UID: \"f4b8f388-11c3-4acb-a008-501c23839953\") " pod="openshift-ingress/e2e-migrate-d6ad08cc-gw-openshift-default-787fcd9587-bs976"
Jul 16 11:28:47.075150 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:47.075123    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f4b8f388-11c3-4acb-a008-501c23839953-istio-envoy\") pod \"e2e-migrate-d6ad08cc-gw-openshift-default-787fcd9587-bs976\" (UID: \"f4b8f388-11c3-4acb-a008-501c23839953\") " pod="openshift-ingress/e2e-migrate-d6ad08cc-gw-openshift-default-787fcd9587-bs976"
Jul 16 11:28:47.075283 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:47.075265    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f4b8f388-11c3-4acb-a008-501c23839953-istio-podinfo\") pod \"e2e-migrate-d6ad08cc-gw-openshift-default-787fcd9587-bs976\" (UID: \"f4b8f388-11c3-4acb-a008-501c23839953\") " pod="openshift-ingress/e2e-migrate-d6ad08cc-gw-openshift-default-787fcd9587-bs976"
Jul 16 11:28:47.082661 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:47.082632    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f4b8f388-11c3-4acb-a008-501c23839953-istio-token\") pod \"e2e-migrate-d6ad08cc-gw-openshift-default-787fcd9587-bs976\" (UID: \"f4b8f388-11c3-4acb-a008-501c23839953\") " pod="openshift-ingress/e2e-migrate-d6ad08cc-gw-openshift-default-787fcd9587-bs976"
Jul 16 11:28:47.082857 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:47.082836    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qkmnz\" (UniqueName: \"kubernetes.io/projected/f4b8f388-11c3-4acb-a008-501c23839953-kube-api-access-qkmnz\") pod \"e2e-migrate-d6ad08cc-gw-openshift-default-787fcd9587-bs976\" (UID: \"f4b8f388-11c3-4acb-a008-501c23839953\") " pod="openshift-ingress/e2e-migrate-d6ad08cc-gw-openshift-default-787fcd9587-bs976"
Jul 16 11:28:47.232968 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:47.232861    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-d6ad08cc-gw-openshift-default-787fcd9587-bs976"
Jul 16 11:28:47.384821 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:47.384791    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-d6ad08cc-gw-openshift-default-787fcd9587-bs976"]
Jul 16 11:28:47.386806 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:28:47.386773    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4b8f388_11c3_4acb_a008_501c23839953.slice/crio-32e410678f6ad391cb7d158b42391344be0bacbc01be632fb08f4fa41709aee0 WatchSource:0}: Error finding container 32e410678f6ad391cb7d158b42391344be0bacbc01be632fb08f4fa41709aee0: Status 404 returned error can't find the container with id 32e410678f6ad391cb7d158b42391344be0bacbc01be632fb08f4fa41709aee0
Jul 16 11:28:47.388872 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:47.388839    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 11:28:47.388970 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:47.388932    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 11:28:47.388970 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:47.388964    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 11:28:47.518192 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:47.518100    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-d6ad08cc-5746cc6996-z6ck7"]
Jul 16 11:28:47.521719 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:47.521696    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d6ad08cc-5746cc6996-z6ck7"
Jul 16 11:28:47.524312 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:47.524292    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-d6ad08cc-serving-cert\""
Jul 16 11:28:47.532705 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:47.532670    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-d6ad08cc-5746cc6996-z6ck7"]
Jul 16 11:28:47.552473 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:47.552376    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-d6ad08cc-gw-openshift-default-787fcd9587-bs976" event={"ID":"f4b8f388-11c3-4acb-a008-501c23839953","Type":"ContainerStarted","Data":"3fe6d88dcfd0be77aceaca70fa72f5b89d29c6c37afaaae11006445dbd7ff1f3"}
Jul 16 11:28:47.552473 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:47.552432    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-d6ad08cc-gw-openshift-default-787fcd9587-bs976" event={"ID":"f4b8f388-11c3-4acb-a008-501c23839953","Type":"ContainerStarted","Data":"32e410678f6ad391cb7d158b42391344be0bacbc01be632fb08f4fa41709aee0"}
Jul 16 11:28:47.577999 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:47.577852    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bzlqs\" (UniqueName: \"kubernetes.io/projected/bc2caea5-08a1-4d04-8951-f4ad28a2082e-kube-api-access-bzlqs\") pod \"maas-api-e2e-migrate-d6ad08cc-5746cc6996-z6ck7\" (UID: \"bc2caea5-08a1-4d04-8951-f4ad28a2082e\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d6ad08cc-5746cc6996-z6ck7"
Jul 16 11:28:47.577999 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:47.577939    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bc2caea5-08a1-4d04-8951-f4ad28a2082e-openshift-service-ca\") pod \"maas-api-e2e-migrate-d6ad08cc-5746cc6996-z6ck7\" (UID: \"bc2caea5-08a1-4d04-8951-f4ad28a2082e\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d6ad08cc-5746cc6996-z6ck7"
Jul 16 11:28:47.578306 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:47.578046    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bc2caea5-08a1-4d04-8951-f4ad28a2082e-maas-api-tls\") pod \"maas-api-e2e-migrate-d6ad08cc-5746cc6996-z6ck7\" (UID: \"bc2caea5-08a1-4d04-8951-f4ad28a2082e\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d6ad08cc-5746cc6996-z6ck7"
Jul 16 11:28:47.578972 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:47.578887    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-d6ad08cc-gw-openshift-default-787fcd9587-bs976" podStartSLOduration=1.578865946 podStartE2EDuration="1.578865946s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 11:28:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 11:28:47.576855412 +0000 UTC m=+1520.211076867" watchObservedRunningTime="2026-07-16 11:28:47.578865946 +0000 UTC m=+1520.213087400"
Jul 16 11:28:47.679272 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:47.679231    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bc2caea5-08a1-4d04-8951-f4ad28a2082e-maas-api-tls\") pod \"maas-api-e2e-migrate-d6ad08cc-5746cc6996-z6ck7\" (UID: \"bc2caea5-08a1-4d04-8951-f4ad28a2082e\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d6ad08cc-5746cc6996-z6ck7"
Jul 16 11:28:47.679487 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:47.679292    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bzlqs\" (UniqueName: \"kubernetes.io/projected/bc2caea5-08a1-4d04-8951-f4ad28a2082e-kube-api-access-bzlqs\") pod \"maas-api-e2e-migrate-d6ad08cc-5746cc6996-z6ck7\" (UID: \"bc2caea5-08a1-4d04-8951-f4ad28a2082e\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d6ad08cc-5746cc6996-z6ck7"
Jul 16 11:28:47.679487 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:47.679367    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bc2caea5-08a1-4d04-8951-f4ad28a2082e-openshift-service-ca\") pod \"maas-api-e2e-migrate-d6ad08cc-5746cc6996-z6ck7\" (UID: \"bc2caea5-08a1-4d04-8951-f4ad28a2082e\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d6ad08cc-5746cc6996-z6ck7"
Jul 16 11:28:47.679487 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:28:47.679418    2583 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-migrate-d6ad08cc-serving-cert: secret "maas-api-e2e-migrate-d6ad08cc-serving-cert" not found
Jul 16 11:28:47.679487 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:28:47.679488    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bc2caea5-08a1-4d04-8951-f4ad28a2082e-maas-api-tls podName:bc2caea5-08a1-4d04-8951-f4ad28a2082e nodeName:}" failed. No retries permitted until 2026-07-16 11:28:48.179465604 +0000 UTC m=+1520.813687048 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/bc2caea5-08a1-4d04-8951-f4ad28a2082e-maas-api-tls") pod "maas-api-e2e-migrate-d6ad08cc-5746cc6996-z6ck7" (UID: "bc2caea5-08a1-4d04-8951-f4ad28a2082e") : secret "maas-api-e2e-migrate-d6ad08cc-serving-cert" not found
Jul 16 11:28:47.680378 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:47.680347    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bc2caea5-08a1-4d04-8951-f4ad28a2082e-openshift-service-ca\") pod \"maas-api-e2e-migrate-d6ad08cc-5746cc6996-z6ck7\" (UID: \"bc2caea5-08a1-4d04-8951-f4ad28a2082e\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d6ad08cc-5746cc6996-z6ck7"
Jul 16 11:28:47.691753 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:47.691718    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bzlqs\" (UniqueName: \"kubernetes.io/projected/bc2caea5-08a1-4d04-8951-f4ad28a2082e-kube-api-access-bzlqs\") pod \"maas-api-e2e-migrate-d6ad08cc-5746cc6996-z6ck7\" (UID: \"bc2caea5-08a1-4d04-8951-f4ad28a2082e\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d6ad08cc-5746cc6996-z6ck7"
Jul 16 11:28:48.185136 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:48.185093    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bc2caea5-08a1-4d04-8951-f4ad28a2082e-maas-api-tls\") pod \"maas-api-e2e-migrate-d6ad08cc-5746cc6996-z6ck7\" (UID: \"bc2caea5-08a1-4d04-8951-f4ad28a2082e\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d6ad08cc-5746cc6996-z6ck7"
Jul 16 11:28:48.187675 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:48.187642    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bc2caea5-08a1-4d04-8951-f4ad28a2082e-maas-api-tls\") pod \"maas-api-e2e-migrate-d6ad08cc-5746cc6996-z6ck7\" (UID: \"bc2caea5-08a1-4d04-8951-f4ad28a2082e\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d6ad08cc-5746cc6996-z6ck7"
Jul 16 11:28:48.233250 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:48.233212    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-d6ad08cc-gw-openshift-default-787fcd9587-bs976"
Jul 16 11:28:48.238738 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:48.238665    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-d6ad08cc-gw-openshift-default-787fcd9587-bs976"
Jul 16 11:28:48.441123 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:48.441032    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d6ad08cc-5746cc6996-z6ck7"
Jul 16 11:28:48.561269 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:48.561218    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-d6ad08cc-gw-openshift-default-787fcd9587-bs976"
Jul 16 11:28:48.562559 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:48.562538    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-d6ad08cc-gw-openshift-default-787fcd9587-bs976"
Jul 16 11:28:48.590414 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:48.590387    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-d6ad08cc-5746cc6996-z6ck7"]
Jul 16 11:28:48.593085 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:28:48.593053    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc2caea5_08a1_4d04_8951_f4ad28a2082e.slice/crio-13a54c7f7ff8b266eceb72e5644f45699ff1a6795af323a94b83e0e949f5d498 WatchSource:0}: Error finding container 13a54c7f7ff8b266eceb72e5644f45699ff1a6795af323a94b83e0e949f5d498: Status 404 returned error can't find the container with id 13a54c7f7ff8b266eceb72e5644f45699ff1a6795af323a94b83e0e949f5d498
Jul 16 11:28:49.567824 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:49.567783    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d6ad08cc-5746cc6996-z6ck7" event={"ID":"bc2caea5-08a1-4d04-8951-f4ad28a2082e","Type":"ContainerStarted","Data":"616d0e6db2497f23aa919544464e1f45bd35e6745376cec7d22e178e78edf099"}
Jul 16 11:28:49.568325 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:49.567851    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d6ad08cc-5746cc6996-z6ck7" event={"ID":"bc2caea5-08a1-4d04-8951-f4ad28a2082e","Type":"ContainerStarted","Data":"13a54c7f7ff8b266eceb72e5644f45699ff1a6795af323a94b83e0e949f5d498"}
Jul 16 11:28:49.568325 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:49.568099    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d6ad08cc-5746cc6996-z6ck7"
Jul 16 11:28:49.590276 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:49.590218    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d6ad08cc-5746cc6996-z6ck7" podStartSLOduration=2.197435603 podStartE2EDuration="2.590202348s" totalImagesPullingTime="392.766745ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 11:28:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 11:28:49.587899506 +0000 UTC m=+1522.222120959" watchObservedRunningTime="2026-07-16 11:28:49.590202348 +0000 UTC m=+1522.224423801"
Jul 16 11:28:52.779302 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:52.779263    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-d6ad08cc-gw-openshift-default-787fcd9587-bs976"]
Jul 16 11:28:52.779740 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:52.779463    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-d6ad08cc-gw-openshift-default-787fcd9587-bs976" podUID="f4b8f388-11c3-4acb-a008-501c23839953" containerName="istio-proxy" containerID="cri-o://3fe6d88dcfd0be77aceaca70fa72f5b89d29c6c37afaaae11006445dbd7ff1f3" gracePeriod=30
Jul 16 11:28:55.578269 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:55.578238    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d6ad08cc-5746cc6996-z6ck7"
Jul 16 11:28:58.080459 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:58.080422    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-d6ad08cc-5746cc6996-z6ck7"]
Jul 16 11:28:58.081026 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:58.080631    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d6ad08cc-5746cc6996-z6ck7" podUID="bc2caea5-08a1-4d04-8951-f4ad28a2082e" containerName="maas-api" containerID="cri-o://616d0e6db2497f23aa919544464e1f45bd35e6745376cec7d22e178e78edf099" gracePeriod=30
Jul 16 11:28:58.344837 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:58.344774    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d6ad08cc-5746cc6996-z6ck7"
Jul 16 11:28:58.488531 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:58.488492    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bc2caea5-08a1-4d04-8951-f4ad28a2082e-openshift-service-ca\") pod \"bc2caea5-08a1-4d04-8951-f4ad28a2082e\" (UID: \"bc2caea5-08a1-4d04-8951-f4ad28a2082e\") "
Jul 16 11:28:58.488750 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:58.488565    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bc2caea5-08a1-4d04-8951-f4ad28a2082e-maas-api-tls\") pod \"bc2caea5-08a1-4d04-8951-f4ad28a2082e\" (UID: \"bc2caea5-08a1-4d04-8951-f4ad28a2082e\") "
Jul 16 11:28:58.488750 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:58.488607    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bzlqs\" (UniqueName: \"kubernetes.io/projected/bc2caea5-08a1-4d04-8951-f4ad28a2082e-kube-api-access-bzlqs\") pod \"bc2caea5-08a1-4d04-8951-f4ad28a2082e\" (UID: \"bc2caea5-08a1-4d04-8951-f4ad28a2082e\") "
Jul 16 11:28:58.488980 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:58.488959    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bc2caea5-08a1-4d04-8951-f4ad28a2082e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "bc2caea5-08a1-4d04-8951-f4ad28a2082e" (UID: "bc2caea5-08a1-4d04-8951-f4ad28a2082e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 11:28:58.490912 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:58.490879    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bc2caea5-08a1-4d04-8951-f4ad28a2082e-kube-api-access-bzlqs" (OuterVolumeSpecName: "kube-api-access-bzlqs") pod "bc2caea5-08a1-4d04-8951-f4ad28a2082e" (UID: "bc2caea5-08a1-4d04-8951-f4ad28a2082e"). InnerVolumeSpecName "kube-api-access-bzlqs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 11:28:58.491045 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:58.490921    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bc2caea5-08a1-4d04-8951-f4ad28a2082e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "bc2caea5-08a1-4d04-8951-f4ad28a2082e" (UID: "bc2caea5-08a1-4d04-8951-f4ad28a2082e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 11:28:58.589403 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:58.589366    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bc2caea5-08a1-4d04-8951-f4ad28a2082e-maas-api-tls\") on node \"ip-10-0-129-28.ec2.internal\" DevicePath \"\""
Jul 16 11:28:58.589403 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:58.589398    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bzlqs\" (UniqueName: \"kubernetes.io/projected/bc2caea5-08a1-4d04-8951-f4ad28a2082e-kube-api-access-bzlqs\") on node \"ip-10-0-129-28.ec2.internal\" DevicePath \"\""
Jul 16 11:28:58.589403 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:58.589408    2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bc2caea5-08a1-4d04-8951-f4ad28a2082e-openshift-service-ca\") on node \"ip-10-0-129-28.ec2.internal\" DevicePath \"\""
Jul 16 11:28:58.613679 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:58.613580    2583 generic.go:356] "Generic (PLEG): container finished" podID="bc2caea5-08a1-4d04-8951-f4ad28a2082e" containerID="616d0e6db2497f23aa919544464e1f45bd35e6745376cec7d22e178e78edf099" exitCode=0
Jul 16 11:28:58.613679 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:58.613663    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d6ad08cc-5746cc6996-z6ck7"
Jul 16 11:28:58.613679 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:58.613667    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d6ad08cc-5746cc6996-z6ck7" event={"ID":"bc2caea5-08a1-4d04-8951-f4ad28a2082e","Type":"ContainerDied","Data":"616d0e6db2497f23aa919544464e1f45bd35e6745376cec7d22e178e78edf099"}
Jul 16 11:28:58.613895 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:58.613726    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d6ad08cc-5746cc6996-z6ck7" event={"ID":"bc2caea5-08a1-4d04-8951-f4ad28a2082e","Type":"ContainerDied","Data":"13a54c7f7ff8b266eceb72e5644f45699ff1a6795af323a94b83e0e949f5d498"}
Jul 16 11:28:58.613895 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:58.613748    2583 scope.go:117] "RemoveContainer" containerID="616d0e6db2497f23aa919544464e1f45bd35e6745376cec7d22e178e78edf099"
Jul 16 11:28:58.626741 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:58.626719    2583 scope.go:117] "RemoveContainer" containerID="616d0e6db2497f23aa919544464e1f45bd35e6745376cec7d22e178e78edf099"
Jul 16 11:28:58.627171 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:28:58.627144    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"616d0e6db2497f23aa919544464e1f45bd35e6745376cec7d22e178e78edf099\": container with ID starting with 616d0e6db2497f23aa919544464e1f45bd35e6745376cec7d22e178e78edf099 not found: ID does not exist" containerID="616d0e6db2497f23aa919544464e1f45bd35e6745376cec7d22e178e78edf099"
Jul 16 11:28:58.627300 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:58.627177    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"616d0e6db2497f23aa919544464e1f45bd35e6745376cec7d22e178e78edf099"} err="failed to get container status \"616d0e6db2497f23aa919544464e1f45bd35e6745376cec7d22e178e78edf099\": rpc error: code = NotFound desc = could not find container \"616d0e6db2497f23aa919544464e1f45bd35e6745376cec7d22e178e78edf099\": container with ID starting with 616d0e6db2497f23aa919544464e1f45bd35e6745376cec7d22e178e78edf099 not found: ID does not exist"
Jul 16 11:28:58.653840 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:58.653798    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-d6ad08cc-5746cc6996-z6ck7"]
Jul 16 11:28:58.663132 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:58.663100    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-d6ad08cc-5746cc6996-z6ck7"]
Jul 16 11:28:59.139863 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:59.139834    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-d6ad08cc-gw-openshift-default-787fcd9587-bs976"
Jul 16 11:28:59.228427 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:59.228344    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bc2caea5-08a1-4d04-8951-f4ad28a2082e" path="/var/lib/kubelet/pods/bc2caea5-08a1-4d04-8951-f4ad28a2082e/volumes"
Jul 16 11:28:59.294625 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:59.294550    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f4b8f388-11c3-4acb-a008-501c23839953-istiod-ca-cert\") pod \"f4b8f388-11c3-4acb-a008-501c23839953\" (UID: \"f4b8f388-11c3-4acb-a008-501c23839953\") "
Jul 16 11:28:59.294832 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:59.294645    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4b8f388-11c3-4acb-a008-501c23839953-workload-socket\") pod \"f4b8f388-11c3-4acb-a008-501c23839953\" (UID: \"f4b8f388-11c3-4acb-a008-501c23839953\") "
Jul 16 11:28:59.294832 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:59.294679    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4b8f388-11c3-4acb-a008-501c23839953-credential-socket\") pod \"f4b8f388-11c3-4acb-a008-501c23839953\" (UID: \"f4b8f388-11c3-4acb-a008-501c23839953\") "
Jul 16 11:28:59.294832 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:59.294697    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f4b8f388-11c3-4acb-a008-501c23839953-istio-podinfo\") pod \"f4b8f388-11c3-4acb-a008-501c23839953\" (UID: \"f4b8f388-11c3-4acb-a008-501c23839953\") "
Jul 16 11:28:59.294832 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:59.294722    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f4b8f388-11c3-4acb-a008-501c23839953-istio-data\") pod \"f4b8f388-11c3-4acb-a008-501c23839953\" (UID: \"f4b8f388-11c3-4acb-a008-501c23839953\") "
Jul 16 11:28:59.294832 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:59.294783    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qkmnz\" (UniqueName: \"kubernetes.io/projected/f4b8f388-11c3-4acb-a008-501c23839953-kube-api-access-qkmnz\") pod \"f4b8f388-11c3-4acb-a008-501c23839953\" (UID: \"f4b8f388-11c3-4acb-a008-501c23839953\") "
Jul 16 11:28:59.294832 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:59.294823    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f4b8f388-11c3-4acb-a008-501c23839953-istio-envoy\") pod \"f4b8f388-11c3-4acb-a008-501c23839953\" (UID: \"f4b8f388-11c3-4acb-a008-501c23839953\") "
Jul 16 11:28:59.295140 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:59.294849    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f4b8f388-11c3-4acb-a008-501c23839953-workload-certs\") pod \"f4b8f388-11c3-4acb-a008-501c23839953\" (UID: \"f4b8f388-11c3-4acb-a008-501c23839953\") "
Jul 16 11:28:59.295140 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:59.294886    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f4b8f388-11c3-4acb-a008-501c23839953-istio-token\") pod \"f4b8f388-11c3-4acb-a008-501c23839953\" (UID: \"f4b8f388-11c3-4acb-a008-501c23839953\") "
Jul 16 11:28:59.295140 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:59.294929    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f4b8f388-11c3-4acb-a008-501c23839953-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f4b8f388-11c3-4acb-a008-501c23839953" (UID: "f4b8f388-11c3-4acb-a008-501c23839953"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 11:28:59.295140 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:59.295014    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f4b8f388-11c3-4acb-a008-501c23839953-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f4b8f388-11c3-4acb-a008-501c23839953" (UID: "f4b8f388-11c3-4acb-a008-501c23839953"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 11:28:59.295140 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:59.295069    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f4b8f388-11c3-4acb-a008-501c23839953-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f4b8f388-11c3-4acb-a008-501c23839953" (UID: "f4b8f388-11c3-4acb-a008-501c23839953"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 11:28:59.295400 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:59.295309    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f4b8f388-11c3-4acb-a008-501c23839953-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f4b8f388-11c3-4acb-a008-501c23839953" (UID: "f4b8f388-11c3-4acb-a008-501c23839953"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 11:28:59.295400 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:59.295328    2583 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f4b8f388-11c3-4acb-a008-501c23839953-istiod-ca-cert\") on node \"ip-10-0-129-28.ec2.internal\" DevicePath \"\""
Jul 16 11:28:59.295400 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:59.295350    2583 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4b8f388-11c3-4acb-a008-501c23839953-workload-socket\") on node \"ip-10-0-129-28.ec2.internal\" DevicePath \"\""
Jul 16 11:28:59.295400 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:59.295363    2583 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4b8f388-11c3-4acb-a008-501c23839953-credential-socket\") on node \"ip-10-0-129-28.ec2.internal\" DevicePath \"\""
Jul 16 11:28:59.295702 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:59.295677    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f4b8f388-11c3-4acb-a008-501c23839953-istio-data" (OuterVolumeSpecName: "istio-data") pod "f4b8f388-11c3-4acb-a008-501c23839953" (UID: "f4b8f388-11c3-4acb-a008-501c23839953"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 11:28:59.297161 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:59.297140    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f4b8f388-11c3-4acb-a008-501c23839953-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f4b8f388-11c3-4acb-a008-501c23839953" (UID: "f4b8f388-11c3-4acb-a008-501c23839953"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 11:28:59.297233 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:59.297192    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f4b8f388-11c3-4acb-a008-501c23839953-istio-token" (OuterVolumeSpecName: "istio-token") pod "f4b8f388-11c3-4acb-a008-501c23839953" (UID: "f4b8f388-11c3-4acb-a008-501c23839953"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 11:28:59.297479 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:59.297453    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f4b8f388-11c3-4acb-a008-501c23839953-kube-api-access-qkmnz" (OuterVolumeSpecName: "kube-api-access-qkmnz") pod "f4b8f388-11c3-4acb-a008-501c23839953" (UID: "f4b8f388-11c3-4acb-a008-501c23839953"). InnerVolumeSpecName "kube-api-access-qkmnz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 11:28:59.297774 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:59.297471    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f4b8f388-11c3-4acb-a008-501c23839953-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f4b8f388-11c3-4acb-a008-501c23839953" (UID: "f4b8f388-11c3-4acb-a008-501c23839953"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 16 11:28:59.395969 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:59.395931    2583 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f4b8f388-11c3-4acb-a008-501c23839953-istio-data\") on node \"ip-10-0-129-28.ec2.internal\" DevicePath \"\""
Jul 16 11:28:59.395969 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:59.395963    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qkmnz\" (UniqueName: \"kubernetes.io/projected/f4b8f388-11c3-4acb-a008-501c23839953-kube-api-access-qkmnz\") on node \"ip-10-0-129-28.ec2.internal\" DevicePath \"\""
Jul 16 11:28:59.395969 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:59.395972    2583 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f4b8f388-11c3-4acb-a008-501c23839953-istio-envoy\") on node \"ip-10-0-129-28.ec2.internal\" DevicePath \"\""
Jul 16 11:28:59.395969 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:59.395981    2583 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f4b8f388-11c3-4acb-a008-501c23839953-workload-certs\") on node \"ip-10-0-129-28.ec2.internal\" DevicePath \"\""
Jul 16 11:28:59.396245 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:59.395990    2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f4b8f388-11c3-4acb-a008-501c23839953-istio-token\") on node \"ip-10-0-129-28.ec2.internal\" DevicePath \"\""
Jul 16 11:28:59.396245 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:59.395998    2583 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f4b8f388-11c3-4acb-a008-501c23839953-istio-podinfo\") on node \"ip-10-0-129-28.ec2.internal\" DevicePath \"\""
Jul 16 11:28:59.620221 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:59.620126    2583 generic.go:356] "Generic (PLEG): container finished" podID="f4b8f388-11c3-4acb-a008-501c23839953" containerID="3fe6d88dcfd0be77aceaca70fa72f5b89d29c6c37afaaae11006445dbd7ff1f3" exitCode=0
Jul 16 11:28:59.620221 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:59.620187    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-d6ad08cc-gw-openshift-default-787fcd9587-bs976"
Jul 16 11:28:59.620463 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:59.620223    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-d6ad08cc-gw-openshift-default-787fcd9587-bs976" event={"ID":"f4b8f388-11c3-4acb-a008-501c23839953","Type":"ContainerDied","Data":"3fe6d88dcfd0be77aceaca70fa72f5b89d29c6c37afaaae11006445dbd7ff1f3"}
Jul 16 11:28:59.620463 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:59.620316    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-d6ad08cc-gw-openshift-default-787fcd9587-bs976" event={"ID":"f4b8f388-11c3-4acb-a008-501c23839953","Type":"ContainerDied","Data":"32e410678f6ad391cb7d158b42391344be0bacbc01be632fb08f4fa41709aee0"}
Jul 16 11:28:59.620463 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:59.620349    2583 scope.go:117] "RemoveContainer" containerID="3fe6d88dcfd0be77aceaca70fa72f5b89d29c6c37afaaae11006445dbd7ff1f3"
Jul 16 11:28:59.629751 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:59.629731    2583 scope.go:117] "RemoveContainer" containerID="3fe6d88dcfd0be77aceaca70fa72f5b89d29c6c37afaaae11006445dbd7ff1f3"
Jul 16 11:28:59.630055 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:28:59.630035    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3fe6d88dcfd0be77aceaca70fa72f5b89d29c6c37afaaae11006445dbd7ff1f3\": container with ID starting with 3fe6d88dcfd0be77aceaca70fa72f5b89d29c6c37afaaae11006445dbd7ff1f3 not found: ID does not exist" containerID="3fe6d88dcfd0be77aceaca70fa72f5b89d29c6c37afaaae11006445dbd7ff1f3"
Jul 16 11:28:59.630125 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:59.630064    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3fe6d88dcfd0be77aceaca70fa72f5b89d29c6c37afaaae11006445dbd7ff1f3"} err="failed to get container status \"3fe6d88dcfd0be77aceaca70fa72f5b89d29c6c37afaaae11006445dbd7ff1f3\": rpc error: code = NotFound desc = could not find container \"3fe6d88dcfd0be77aceaca70fa72f5b89d29c6c37afaaae11006445dbd7ff1f3\": container with ID starting with 3fe6d88dcfd0be77aceaca70fa72f5b89d29c6c37afaaae11006445dbd7ff1f3 not found: ID does not exist"
Jul 16 11:28:59.645757 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:59.645730    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-d6ad08cc-gw-openshift-default-787fcd9587-bs976"]
Jul 16 11:28:59.652375 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:28:59.652351    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-d6ad08cc-gw-openshift-default-787fcd9587-bs976"]
Jul 16 11:29:01.229480 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:29:01.229445    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f4b8f388-11c3-4acb-a008-501c23839953" path="/var/lib/kubelet/pods/f4b8f388-11c3-4acb-a008-501c23839953/volumes"
Jul 16 11:29:05.360101 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:29:05.360065    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-11372ed9-openshift-default-dc4cd7d99-tv64h"]
Jul 16 11:29:05.360641 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:29:05.360622    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f4b8f388-11c3-4acb-a008-501c23839953" containerName="istio-proxy"
Jul 16 11:29:05.360641 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:29:05.360642    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="f4b8f388-11c3-4acb-a008-501c23839953" containerName="istio-proxy"
Jul 16 11:29:05.360784 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:29:05.360664    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bc2caea5-08a1-4d04-8951-f4ad28a2082e" containerName="maas-api"
Jul 16 11:29:05.360784 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:29:05.360674    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="bc2caea5-08a1-4d04-8951-f4ad28a2082e" containerName="maas-api"
Jul 16 11:29:05.360784 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:29:05.360765    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="bc2caea5-08a1-4d04-8951-f4ad28a2082e" containerName="maas-api"
Jul 16 11:29:05.360784 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:29:05.360785    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="f4b8f388-11c3-4acb-a008-501c23839953" containerName="istio-proxy"
Jul 16 11:29:05.364186 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:29:05.364157    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-11372ed9-openshift-default-dc4cd7d99-tv64h"
Jul 16 11:29:05.368460 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:29:05.368435    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-11372ed9-openshift-default-dockercfg-shxwl\""
Jul 16 11:29:05.378129 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:29:05.378096    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-11372ed9-openshift-default-dc4cd7d99-tv64h"]
Jul 16 11:29:05.447245 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:29:05.447206    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vtj64\" (UniqueName: \"kubernetes.io/projected/744e010e-fd4b-4535-ae6c-4e1e9680b073-kube-api-access-vtj64\") pod \"e2e-derive-11372ed9-openshift-default-dc4cd7d99-tv64h\" (UID: \"744e010e-fd4b-4535-ae6c-4e1e9680b073\") " pod="openshift-ingress/e2e-derive-11372ed9-openshift-default-dc4cd7d99-tv64h"
Jul 16 11:29:05.447455 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:29:05.447274    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/744e010e-fd4b-4535-ae6c-4e1e9680b073-workload-certs\") pod \"e2e-derive-11372ed9-openshift-default-dc4cd7d99-tv64h\" (UID: \"744e010e-fd4b-4535-ae6c-4e1e9680b073\") " pod="openshift-ingress/e2e-derive-11372ed9-openshift-default-dc4cd7d99-tv64h"
Jul 16 11:29:05.447455 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:29:05.447344    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/744e010e-fd4b-4535-ae6c-4e1e9680b073-istiod-ca-cert\") pod \"e2e-derive-11372ed9-openshift-default-dc4cd7d99-tv64h\" (UID: \"744e010e-fd4b-4535-ae6c-4e1e9680b073\") " pod="openshift-ingress/e2e-derive-11372ed9-openshift-default-dc4cd7d99-tv64h"
Jul 16 11:29:05.447455 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:29:05.447384    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/744e010e-fd4b-4535-ae6c-4e1e9680b073-istio-token\") pod \"e2e-derive-11372ed9-openshift-default-dc4cd7d99-tv64h\" (UID: \"744e010e-fd4b-4535-ae6c-4e1e9680b073\") " pod="openshift-ingress/e2e-derive-11372ed9-openshift-default-dc4cd7d99-tv64h"
Jul 16 11:29:05.447455 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:29:05.447423    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/744e010e-fd4b-4535-ae6c-4e1e9680b073-workload-socket\") pod \"e2e-derive-11372ed9-openshift-default-dc4cd7d99-tv64h\" (UID: \"744e010e-fd4b-4535-ae6c-4e1e9680b073\") " pod="openshift-ingress/e2e-derive-11372ed9-openshift-default-dc4cd7d99-tv64h"
Jul 16 11:29:05.447725 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:29:05.447460    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/744e010e-fd4b-4535-ae6c-4e1e9680b073-credential-socket\") pod \"e2e-derive-11372ed9-openshift-default-dc4cd7d99-tv64h\" (UID: \"744e010e-fd4b-4535-ae6c-4e1e9680b073\") " pod="openshift-ingress/e2e-derive-11372ed9-openshift-default-dc4cd7d99-tv64h"
Jul 16 11:29:05.447725 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:29:05.447488    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/744e010e-fd4b-4535-ae6c-4e1e9680b073-istio-podinfo\") pod \"e2e-derive-11372ed9-openshift-default-dc4cd7d99-tv64h\" (UID: \"744e010e-fd4b-4535-ae6c-4e1e9680b073\") " pod="openshift-ingress/e2e-derive-11372ed9-openshift-default-dc4cd7d99-tv64h"
Jul 16 11:29:05.447725 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:29:05.447538    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/744e010e-fd4b-4535-ae6c-4e1e9680b073-istio-data\") pod \"e2e-derive-11372ed9-openshift-default-dc4cd7d99-tv64h\" (UID: \"744e010e-fd4b-4535-ae6c-4e1e9680b073\") " pod="openshift-ingress/e2e-derive-11372ed9-openshift-default-dc4cd7d99-tv64h"
Jul 16 11:29:05.447725 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:29:05.447651    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/744e010e-fd4b-4535-ae6c-4e1e9680b073-istio-envoy\") pod \"e2e-derive-11372ed9-openshift-default-dc4cd7d99-tv64h\" (UID: \"744e010e-fd4b-4535-ae6c-4e1e9680b073\") " pod="openshift-ingress/e2e-derive-11372ed9-openshift-default-dc4cd7d99-tv64h"
Jul 16 11:29:05.548807 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:29:05.548767    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/744e010e-fd4b-4535-ae6c-4e1e9680b073-istiod-ca-cert\") pod \"e2e-derive-11372ed9-openshift-default-dc4cd7d99-tv64h\" (UID: \"744e010e-fd4b-4535-ae6c-4e1e9680b073\") " pod="openshift-ingress/e2e-derive-11372ed9-openshift-default-dc4cd7d99-tv64h"
Jul 16 11:29:05.548807 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:29:05.548806    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/744e010e-fd4b-4535-ae6c-4e1e9680b073-istio-token\") pod \"e2e-derive-11372ed9-openshift-default-dc4cd7d99-tv64h\" (UID: \"744e010e-fd4b-4535-ae6c-4e1e9680b073\") " pod="openshift-ingress/e2e-derive-11372ed9-openshift-default-dc4cd7d99-tv64h"
Jul 16 11:29:05.549083 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:29:05.548827    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/744e010e-fd4b-4535-ae6c-4e1e9680b073-workload-socket\") pod \"e2e-derive-11372ed9-openshift-default-dc4cd7d99-tv64h\" (UID: \"744e010e-fd4b-4535-ae6c-4e1e9680b073\") " pod="openshift-ingress/e2e-derive-11372ed9-openshift-default-dc4cd7d99-tv64h"
Jul 16 11:29:05.549083 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:29:05.548847    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/744e010e-fd4b-4535-ae6c-4e1e9680b073-credential-socket\") pod \"e2e-derive-11372ed9-openshift-default-dc4cd7d99-tv64h\" (UID: \"744e010e-fd4b-4535-ae6c-4e1e9680b073\") " pod="openshift-ingress/e2e-derive-11372ed9-openshift-default-dc4cd7d99-tv64h"
Jul 16 11:29:05.549083 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:29:05.548875    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/744e010e-fd4b-4535-ae6c-4e1e9680b073-istio-podinfo\") pod \"e2e-derive-11372ed9-openshift-default-dc4cd7d99-tv64h\" (UID: \"744e010e-fd4b-4535-ae6c-4e1e9680b073\") " pod="openshift-ingress/e2e-derive-11372ed9-openshift-default-dc4cd7d99-tv64h"
Jul 16 11:29:05.549083 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:29:05.548917    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/744e010e-fd4b-4535-ae6c-4e1e9680b073-istio-data\") pod \"e2e-derive-11372ed9-openshift-default-dc4cd7d99-tv64h\" (UID: \"744e010e-fd4b-4535-ae6c-4e1e9680b073\") " pod="openshift-ingress/e2e-derive-11372ed9-openshift-default-dc4cd7d99-tv64h"
Jul 16 11:29:05.549083 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:29:05.548979    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/744e010e-fd4b-4535-ae6c-4e1e9680b073-istio-envoy\") pod \"e2e-derive-11372ed9-openshift-default-dc4cd7d99-tv64h\" (UID: \"744e010e-fd4b-4535-ae6c-4e1e9680b073\") " pod="openshift-ingress/e2e-derive-11372ed9-openshift-default-dc4cd7d99-tv64h"
Jul 16 11:29:05.549083 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:29:05.549003    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vtj64\" (UniqueName: \"kubernetes.io/projected/744e010e-fd4b-4535-ae6c-4e1e9680b073-kube-api-access-vtj64\") pod \"e2e-derive-11372ed9-openshift-default-dc4cd7d99-tv64h\" (UID: \"744e010e-fd4b-4535-ae6c-4e1e9680b073\") " pod="openshift-ingress/e2e-derive-11372ed9-openshift-default-dc4cd7d99-tv64h"
Jul 16 11:29:05.549083 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:29:05.549073    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/744e010e-fd4b-4535-ae6c-4e1e9680b073-workload-certs\") pod \"e2e-derive-11372ed9-openshift-default-dc4cd7d99-tv64h\" (UID: \"744e010e-fd4b-4535-ae6c-4e1e9680b073\") " pod="openshift-ingress/e2e-derive-11372ed9-openshift-default-dc4cd7d99-tv64h"
Jul 16 11:29:05.549520 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:29:05.549341    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/744e010e-fd4b-4535-ae6c-4e1e9680b073-workload-socket\") pod \"e2e-derive-11372ed9-openshift-default-dc4cd7d99-tv64h\" (UID: \"744e010e-fd4b-4535-ae6c-4e1e9680b073\") " pod="openshift-ingress/e2e-derive-11372ed9-openshift-default-dc4cd7d99-tv64h"
Jul 16 11:29:05.549520 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:29:05.549461    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/744e010e-fd4b-4535-ae6c-4e1e9680b073-credential-socket\") pod \"e2e-derive-11372ed9-openshift-default-dc4cd7d99-tv64h\" (UID: \"744e010e-fd4b-4535-ae6c-4e1e9680b073\") " pod="openshift-ingress/e2e-derive-11372ed9-openshift-default-dc4cd7d99-tv64h"
Jul 16 11:29:05.549520 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:29:05.549484    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/744e010e-fd4b-4535-ae6c-4e1e9680b073-workload-certs\") pod \"e2e-derive-11372ed9-openshift-default-dc4cd7d99-tv64h\" (UID: \"744e010e-fd4b-4535-ae6c-4e1e9680b073\") " pod="openshift-ingress/e2e-derive-11372ed9-openshift-default-dc4cd7d99-tv64h"
Jul 16 11:29:05.549729 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:29:05.549680    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/744e010e-fd4b-4535-ae6c-4e1e9680b073-istiod-ca-cert\") pod \"e2e-derive-11372ed9-openshift-default-dc4cd7d99-tv64h\" (UID: \"744e010e-fd4b-4535-ae6c-4e1e9680b073\") " pod="openshift-ingress/e2e-derive-11372ed9-openshift-default-dc4cd7d99-tv64h"
Jul 16 11:29:05.549729 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:29:05.549707    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/744e010e-fd4b-4535-ae6c-4e1e9680b073-istio-data\") pod \"e2e-derive-11372ed9-openshift-default-dc4cd7d99-tv64h\" (UID: \"744e010e-fd4b-4535-ae6c-4e1e9680b073\") " pod="openshift-ingress/e2e-derive-11372ed9-openshift-default-dc4cd7d99-tv64h"
Jul 16 11:29:05.551354 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:29:05.551331    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/744e010e-fd4b-4535-ae6c-4e1e9680b073-istio-envoy\") pod \"e2e-derive-11372ed9-openshift-default-dc4cd7d99-tv64h\" (UID: \"744e010e-fd4b-4535-ae6c-4e1e9680b073\") " pod="openshift-ingress/e2e-derive-11372ed9-openshift-default-dc4cd7d99-tv64h"
Jul 16 11:29:05.551496 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:29:05.551478    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/744e010e-fd4b-4535-ae6c-4e1e9680b073-istio-podinfo\") pod \"e2e-derive-11372ed9-openshift-default-dc4cd7d99-tv64h\" (UID: \"744e010e-fd4b-4535-ae6c-4e1e9680b073\") " pod="openshift-ingress/e2e-derive-11372ed9-openshift-default-dc4cd7d99-tv64h"
Jul 16 11:29:05.557560 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:29:05.557536    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/744e010e-fd4b-4535-ae6c-4e1e9680b073-istio-token\") pod \"e2e-derive-11372ed9-openshift-default-dc4cd7d99-tv64h\" (UID: \"744e010e-fd4b-4535-ae6c-4e1e9680b073\") " pod="openshift-ingress/e2e-derive-11372ed9-openshift-default-dc4cd7d99-tv64h"
Jul 16 11:29:05.558373 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:29:05.558350    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vtj64\" (UniqueName: \"kubernetes.io/projected/744e010e-fd4b-4535-ae6c-4e1e9680b073-kube-api-access-vtj64\") pod \"e2e-derive-11372ed9-openshift-default-dc4cd7d99-tv64h\" (UID: \"744e010e-fd4b-4535-ae6c-4e1e9680b073\") " pod="openshift-ingress/e2e-derive-11372ed9-openshift-default-dc4cd7d99-tv64h"
Jul 16 11:29:05.634127 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:29:05.634038    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-11372ed9-6c9bc5744f-8ncgj"]
Jul 16 11:29:05.638115 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:29:05.638083    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-11372ed9-6c9bc5744f-8ncgj"
Jul 16 11:29:05.640564 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:29:05.640534    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-11372ed9-serving-cert\""
Jul 16 11:29:05.651807 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:29:05.651778    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-11372ed9-6c9bc5744f-8ncgj"]
Jul 16 11:29:05.677399 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:29:05.677357    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-11372ed9-openshift-default-dc4cd7d99-tv64h"
Jul 16 11:29:05.751534 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:29:05.751414    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5f381ead-1bd5-4f97-a751-a232c8b07c83-maas-api-tls\") pod \"maas-api-e2e-derive-11372ed9-6c9bc5744f-8ncgj\" (UID: \"5f381ead-1bd5-4f97-a751-a232c8b07c83\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-11372ed9-6c9bc5744f-8ncgj"
Jul 16 11:29:05.751534 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:29:05.751475    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-77gdk\" (UniqueName: \"kubernetes.io/projected/5f381ead-1bd5-4f97-a751-a232c8b07c83-kube-api-access-77gdk\") pod \"maas-api-e2e-derive-11372ed9-6c9bc5744f-8ncgj\" (UID: \"5f381ead-1bd5-4f97-a751-a232c8b07c83\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-11372ed9-6c9bc5744f-8ncgj"
Jul 16 11:29:05.751920 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:29:05.751578    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f381ead-1bd5-4f97-a751-a232c8b07c83-openshift-service-ca\") pod \"maas-api-e2e-derive-11372ed9-6c9bc5744f-8ncgj\" (UID: \"5f381ead-1bd5-4f97-a751-a232c8b07c83\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-11372ed9-6c9bc5744f-8ncgj"
Jul 16 11:29:05.834553 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:29:05.834526    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-11372ed9-openshift-default-dc4cd7d99-tv64h"]
Jul 16 11:29:05.836970 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:29:05.836941    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod744e010e_fd4b_4535_ae6c_4e1e9680b073.slice/crio-0209ed8c9c8fa9d6eda80675ff900726c276ee9b2b146fb388ae481978c8833b WatchSource:0}: Error finding container 0209ed8c9c8fa9d6eda80675ff900726c276ee9b2b146fb388ae481978c8833b: Status 404 returned error can't find the container with id 0209ed8c9c8fa9d6eda80675ff900726c276ee9b2b146fb388ae481978c8833b
Jul 16 11:29:05.839502 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:29:05.839469    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 11:29:05.840055 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:29:05.839540    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 11:29:05.840055 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:29:05.839570    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 11:29:05.852584 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:29:05.852553    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5f381ead-1bd5-4f97-a751-a232c8b07c83-maas-api-tls\") pod \"maas-api-e2e-derive-11372ed9-6c9bc5744f-8ncgj\" (UID: \"5f381ead-1bd5-4f97-a751-a232c8b07c83\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-11372ed9-6c9bc5744f-8ncgj"
Jul 16 11:29:05.852700 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:29:05.852632    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-77gdk\" (UniqueName: \"kubernetes.io/projected/5f381ead-1bd5-4f97-a751-a232c8b07c83-kube-api-access-77gdk\") pod \"maas-api-e2e-derive-11372ed9-6c9bc5744f-8ncgj\" (UID: \"5f381ead-1bd5-4f97-a751-a232c8b07c83\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-11372ed9-6c9bc5744f-8ncgj"
Jul 16 11:29:05.852758 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:29:05.852724    2583 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-11372ed9-serving-cert: secret "maas-api-e2e-derive-11372ed9-serving-cert" not found
Jul 16 11:29:05.852793 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:29:05.852762    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f381ead-1bd5-4f97-a751-a232c8b07c83-openshift-service-ca\") pod \"maas-api-e2e-derive-11372ed9-6c9bc5744f-8ncgj\" (UID: \"5f381ead-1bd5-4f97-a751-a232c8b07c83\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-11372ed9-6c9bc5744f-8ncgj"
Jul 16 11:29:05.852829 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:29:05.852792    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5f381ead-1bd5-4f97-a751-a232c8b07c83-maas-api-tls podName:5f381ead-1bd5-4f97-a751-a232c8b07c83 nodeName:}" failed. No retries permitted until 2026-07-16 11:29:06.352771199 +0000 UTC m=+1538.986992631 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/5f381ead-1bd5-4f97-a751-a232c8b07c83-maas-api-tls") pod "maas-api-e2e-derive-11372ed9-6c9bc5744f-8ncgj" (UID: "5f381ead-1bd5-4f97-a751-a232c8b07c83") : secret "maas-api-e2e-derive-11372ed9-serving-cert" not found
Jul 16 11:29:05.853382 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:29:05.853361    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f381ead-1bd5-4f97-a751-a232c8b07c83-openshift-service-ca\") pod \"maas-api-e2e-derive-11372ed9-6c9bc5744f-8ncgj\" (UID: \"5f381ead-1bd5-4f97-a751-a232c8b07c83\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-11372ed9-6c9bc5744f-8ncgj"
Jul 16 11:29:05.862731 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:29:05.862702    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-77gdk\" (UniqueName: \"kubernetes.io/projected/5f381ead-1bd5-4f97-a751-a232c8b07c83-kube-api-access-77gdk\") pod \"maas-api-e2e-derive-11372ed9-6c9bc5744f-8ncgj\" (UID: \"5f381ead-1bd5-4f97-a751-a232c8b07c83\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-11372ed9-6c9bc5744f-8ncgj"
Jul 16 11:29:06.358807 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:29:06.358766    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5f381ead-1bd5-4f97-a751-a232c8b07c83-maas-api-tls\") pod \"maas-api-e2e-derive-11372ed9-6c9bc5744f-8ncgj\" (UID: \"5f381ead-1bd5-4f97-a751-a232c8b07c83\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-11372ed9-6c9bc5744f-8ncgj"
Jul 16 11:29:06.361254 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:29:06.361231    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5f381ead-1bd5-4f97-a751-a232c8b07c83-maas-api-tls\") pod \"maas-api-e2e-derive-11372ed9-6c9bc5744f-8ncgj\" (UID: \"5f381ead-1bd5-4f97-a751-a232c8b07c83\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-11372ed9-6c9bc5744f-8ncgj"
Jul 16 11:29:06.551875 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:29:06.551827    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-11372ed9-6c9bc5744f-8ncgj"
Jul 16 11:29:06.661862 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:29:06.661822    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-11372ed9-openshift-default-dc4cd7d99-tv64h" event={"ID":"744e010e-fd4b-4535-ae6c-4e1e9680b073","Type":"ContainerStarted","Data":"d84382d6bca2dcae7946fa42c8a01127ec95491f8b118c4764069dc8074112c6"}
Jul 16 11:29:06.662027 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:29:06.661887    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-11372ed9-openshift-default-dc4cd7d99-tv64h" event={"ID":"744e010e-fd4b-4535-ae6c-4e1e9680b073","Type":"ContainerStarted","Data":"0209ed8c9c8fa9d6eda80675ff900726c276ee9b2b146fb388ae481978c8833b"}
Jul 16 11:29:06.677826 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:29:06.677790    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-11372ed9-openshift-default-dc4cd7d99-tv64h"
Jul 16 11:29:06.682696 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:29:06.682669    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-11372ed9-openshift-default-dc4cd7d99-tv64h"
Jul 16 11:29:06.687029 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:29:06.686974    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-11372ed9-openshift-default-dc4cd7d99-tv64h" podStartSLOduration=1.686952733 podStartE2EDuration="1.686952733s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 11:29:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 11:29:06.685294246 +0000 UTC m=+1539.319515701" watchObservedRunningTime="2026-07-16 11:29:06.686952733 +0000 UTC m=+1539.321174188"
Jul 16 11:29:06.696045 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:29:06.695799    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-11372ed9-6c9bc5744f-8ncgj"]
Jul 16 11:29:06.698840 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:29:06.698805    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f381ead_1bd5_4f97_a751_a232c8b07c83.slice/crio-1f9c3c144fa134a6761e170f79a8624ec8991f9dcb5e55dd60c611075fb81bf3 WatchSource:0}: Error finding container 1f9c3c144fa134a6761e170f79a8624ec8991f9dcb5e55dd60c611075fb81bf3: Status 404 returned error can't find the container with id 1f9c3c144fa134a6761e170f79a8624ec8991f9dcb5e55dd60c611075fb81bf3
Jul 16 11:29:07.667747 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:29:07.667711    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-11372ed9-6c9bc5744f-8ncgj" event={"ID":"5f381ead-1bd5-4f97-a751-a232c8b07c83","Type":"ContainerStarted","Data":"1f9c3c144fa134a6761e170f79a8624ec8991f9dcb5e55dd60c611075fb81bf3"}
Jul 16 11:29:07.668139 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:29:07.667994    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-11372ed9-openshift-default-dc4cd7d99-tv64h"
Jul 16 11:29:07.669091 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:29:07.669072    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-11372ed9-openshift-default-dc4cd7d99-tv64h"
Jul 16 11:29:08.673734 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:29:08.673690    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-11372ed9-6c9bc5744f-8ncgj" event={"ID":"5f381ead-1bd5-4f97-a751-a232c8b07c83","Type":"ContainerStarted","Data":"643224b636707950a34009ed2a9715f1af5aef6d12c1a57526f743032844ab06"}
Jul 16 11:29:08.692892 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:29:08.692843    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-11372ed9-6c9bc5744f-8ncgj" podStartSLOduration=1.937459609 podStartE2EDuration="3.692826706s" totalImagesPullingTime="1.755367097s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 11:29:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 11:29:08.691212595 +0000 UTC m=+1541.325434049" watchObservedRunningTime="2026-07-16 11:29:08.692826706 +0000 UTC m=+1541.327048159"
Jul 16 11:29:09.678322 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:29:09.678282    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-11372ed9-6c9bc5744f-8ncgj"
Jul 16 11:29:11.066108 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:29:11.066069    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-11372ed9-openshift-default-dc4cd7d99-tv64h"]
Jul 16 11:29:11.066514 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:29:11.066262    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-11372ed9-openshift-default-dc4cd7d99-tv64h" podUID="744e010e-fd4b-4535-ae6c-4e1e9680b073" containerName="istio-proxy" containerID="cri-o://d84382d6bca2dcae7946fa42c8a01127ec95491f8b118c4764069dc8074112c6" gracePeriod=30
Jul 16 11:29:15.688512 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:29:15.688481    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-11372ed9-6c9bc5744f-8ncgj"
Jul 16 11:29:16.318141 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:29:16.318114    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-11372ed9-openshift-default-dc4cd7d99-tv64h"
Jul 16 11:29:16.345453 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:29:16.345371    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/744e010e-fd4b-4535-ae6c-4e1e9680b073-workload-certs\") pod \"744e010e-fd4b-4535-ae6c-4e1e9680b073\" (UID: \"744e010e-fd4b-4535-ae6c-4e1e9680b073\") "
Jul 16 11:29:16.345453 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:29:16.345407    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/744e010e-fd4b-4535-ae6c-4e1e9680b073-istio-envoy\") pod \"744e010e-fd4b-4535-ae6c-4e1e9680b073\" (UID: \"744e010e-fd4b-4535-ae6c-4e1e9680b073\") "
Jul 16 11:29:16.345453 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:29:16.345434    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/744e010e-fd4b-4535-ae6c-4e1e9680b073-istio-podinfo\") pod \"744e010e-fd4b-4535-ae6c-4e1e9680b073\" (UID: \"744e010e-fd4b-4535-ae6c-4e1e9680b073\") "
Jul 16 11:29:16.345453 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:29:16.345455    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/744e010e-fd4b-4535-ae6c-4e1e9680b073-istio-token\") pod \"744e010e-fd4b-4535-ae6c-4e1e9680b073\" (UID: \"744e010e-fd4b-4535-ae6c-4e1e9680b073\") "
Jul 16 11:29:16.345750 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:29:16.345485    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vtj64\" (UniqueName: \"kubernetes.io/projected/744e010e-fd4b-4535-ae6c-4e1e9680b073-kube-api-access-vtj64\") pod \"744e010e-fd4b-4535-ae6c-4e1e9680b073\" (UID: \"744e010e-fd4b-4535-ae6c-4e1e9680b073\") "
Jul 16 11:29:16.345750 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:29:16.345505    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/744e010e-fd4b-4535-ae6c-4e1e9680b073-istiod-ca-cert\") pod \"744e010e-fd4b-4535-ae6c-4e1e9680b073\" (UID: \"744e010e-fd4b-4535-ae6c-4e1e9680b073\") "
Jul 16 11:29:16.345750 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:29:16.345523    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/744e010e-fd4b-4535-ae6c-4e1e9680b073-workload-socket\") pod \"744e010e-fd4b-4535-ae6c-4e1e9680b073\" (UID: \"744e010e-fd4b-4535-ae6c-4e1e9680b073\") "
Jul 16 11:29:16.345750 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:29:16.345703    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/744e010e-fd4b-4535-ae6c-4e1e9680b073-credential-socket\") pod \"744e010e-fd4b-4535-ae6c-4e1e9680b073\" (UID: \"744e010e-fd4b-4535-ae6c-4e1e9680b073\") "
Jul 16 11:29:16.345750 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:29:16.345737    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/744e010e-fd4b-4535-ae6c-4e1e9680b073-istio-data\") pod \"744e010e-fd4b-4535-ae6c-4e1e9680b073\" (UID: \"744e010e-fd4b-4535-ae6c-4e1e9680b073\") "
Jul 16 11:29:16.346368 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:29:16.345733    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/744e010e-fd4b-4535-ae6c-4e1e9680b073-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "744e010e-fd4b-4535-ae6c-4e1e9680b073" (UID: "744e010e-fd4b-4535-ae6c-4e1e9680b073"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 11:29:16.346368 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:29:16.345987    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/744e010e-fd4b-4535-ae6c-4e1e9680b073-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "744e010e-fd4b-4535-ae6c-4e1e9680b073" (UID: "744e010e-fd4b-4535-ae6c-4e1e9680b073"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 11:29:16.346368 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:29:16.346149    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/744e010e-fd4b-4535-ae6c-4e1e9680b073-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "744e010e-fd4b-4535-ae6c-4e1e9680b073" (UID: "744e010e-fd4b-4535-ae6c-4e1e9680b073"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 11:29:16.346368 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:29:16.346325    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/744e010e-fd4b-4535-ae6c-4e1e9680b073-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "744e010e-fd4b-4535-ae6c-4e1e9680b073" (UID: "744e010e-fd4b-4535-ae6c-4e1e9680b073"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 11:29:16.346740 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:29:16.346583    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/744e010e-fd4b-4535-ae6c-4e1e9680b073-istio-data" (OuterVolumeSpecName: "istio-data") pod "744e010e-fd4b-4535-ae6c-4e1e9680b073" (UID: "744e010e-fd4b-4535-ae6c-4e1e9680b073"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 11:29:16.348089 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:29:16.348048    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/744e010e-fd4b-4535-ae6c-4e1e9680b073-kube-api-access-vtj64" (OuterVolumeSpecName: "kube-api-access-vtj64") pod "744e010e-fd4b-4535-ae6c-4e1e9680b073" (UID: "744e010e-fd4b-4535-ae6c-4e1e9680b073"). InnerVolumeSpecName "kube-api-access-vtj64". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 11:29:16.348302 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:29:16.348276    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/744e010e-fd4b-4535-ae6c-4e1e9680b073-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "744e010e-fd4b-4535-ae6c-4e1e9680b073" (UID: "744e010e-fd4b-4535-ae6c-4e1e9680b073"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 11:29:16.348404 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:29:16.348380    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/744e010e-fd4b-4535-ae6c-4e1e9680b073-istio-token" (OuterVolumeSpecName: "istio-token") pod "744e010e-fd4b-4535-ae6c-4e1e9680b073" (UID: "744e010e-fd4b-4535-ae6c-4e1e9680b073"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 11:29:16.348521 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:29:16.348494    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/744e010e-fd4b-4535-ae6c-4e1e9680b073-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "744e010e-fd4b-4535-ae6c-4e1e9680b073" (UID: "744e010e-fd4b-4535-ae6c-4e1e9680b073"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 16 11:29:16.446666 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:29:16.446622    2583 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/744e010e-fd4b-4535-ae6c-4e1e9680b073-workload-socket\") on node \"ip-10-0-129-28.ec2.internal\" DevicePath \"\""
Jul 16 11:29:16.446666 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:29:16.446658    2583 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/744e010e-fd4b-4535-ae6c-4e1e9680b073-credential-socket\") on node \"ip-10-0-129-28.ec2.internal\" DevicePath \"\""
Jul 16 11:29:16.446666 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:29:16.446668    2583 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/744e010e-fd4b-4535-ae6c-4e1e9680b073-istio-data\") on node \"ip-10-0-129-28.ec2.internal\" DevicePath \"\""
Jul 16 11:29:16.446666 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:29:16.446676    2583 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/744e010e-fd4b-4535-ae6c-4e1e9680b073-workload-certs\") on node \"ip-10-0-129-28.ec2.internal\" DevicePath \"\""
Jul 16 11:29:16.446943 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:29:16.446686    2583 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/744e010e-fd4b-4535-ae6c-4e1e9680b073-istio-envoy\") on node \"ip-10-0-129-28.ec2.internal\" DevicePath \"\""
Jul 16 11:29:16.446943 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:29:16.446694    2583 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/744e010e-fd4b-4535-ae6c-4e1e9680b073-istio-podinfo\") on node \"ip-10-0-129-28.ec2.internal\" DevicePath \"\""
Jul 16 11:29:16.446943 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:29:16.446701    2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/744e010e-fd4b-4535-ae6c-4e1e9680b073-istio-token\") on node \"ip-10-0-129-28.ec2.internal\" DevicePath \"\""
Jul 16 11:29:16.446943 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:29:16.446709    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vtj64\" (UniqueName: \"kubernetes.io/projected/744e010e-fd4b-4535-ae6c-4e1e9680b073-kube-api-access-vtj64\") on node \"ip-10-0-129-28.ec2.internal\" DevicePath \"\""
Jul 16 11:29:16.446943 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:29:16.446719    2583 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/744e010e-fd4b-4535-ae6c-4e1e9680b073-istiod-ca-cert\") on node \"ip-10-0-129-28.ec2.internal\" DevicePath \"\""
Jul 16 11:29:16.549378 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:29:16.549343    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-11372ed9-6c9bc5744f-8ncgj"]
Jul 16 11:29:16.549659 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:29:16.549612    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-11372ed9-6c9bc5744f-8ncgj" podUID="5f381ead-1bd5-4f97-a751-a232c8b07c83" containerName="maas-api" containerID="cri-o://643224b636707950a34009ed2a9715f1af5aef6d12c1a57526f743032844ab06" gracePeriod=30
Jul 16 11:29:16.714131 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:29:16.714093    2583 generic.go:356] "Generic (PLEG): container finished" podID="744e010e-fd4b-4535-ae6c-4e1e9680b073" containerID="d84382d6bca2dcae7946fa42c8a01127ec95491f8b118c4764069dc8074112c6" exitCode=0
Jul 16 11:29:16.714616 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:29:16.714141    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-11372ed9-openshift-default-dc4cd7d99-tv64h"
Jul 16 11:29:16.714616 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:29:16.714182    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-11372ed9-openshift-default-dc4cd7d99-tv64h" event={"ID":"744e010e-fd4b-4535-ae6c-4e1e9680b073","Type":"ContainerDied","Data":"d84382d6bca2dcae7946fa42c8a01127ec95491f8b118c4764069dc8074112c6"}
Jul 16 11:29:16.714616 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:29:16.714240    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-11372ed9-openshift-default-dc4cd7d99-tv64h" event={"ID":"744e010e-fd4b-4535-ae6c-4e1e9680b073","Type":"ContainerDied","Data":"0209ed8c9c8fa9d6eda80675ff900726c276ee9b2b146fb388ae481978c8833b"}
Jul 16 11:29:16.714616 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:29:16.714270    2583 scope.go:117] "RemoveContainer" containerID="d84382d6bca2dcae7946fa42c8a01127ec95491f8b118c4764069dc8074112c6"
Jul 16 11:29:16.717225 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:29:16.717199    2583 generic.go:356] "Generic (PLEG): container finished" podID="5f381ead-1bd5-4f97-a751-a232c8b07c83" containerID="643224b636707950a34009ed2a9715f1af5aef6d12c1a57526f743032844ab06" exitCode=0
Jul 16 11:29:16.717379 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:29:16.717271    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-11372ed9-6c9bc5744f-8ncgj" event={"ID":"5f381ead-1bd5-4f97-a751-a232c8b07c83","Type":"ContainerDied","Data":"643224b636707950a34009ed2a9715f1af5aef6d12c1a57526f743032844ab06"}
Jul 16 11:29:16.724736 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:29:16.724708    2583 scope.go:117] "RemoveContainer" containerID="d84382d6bca2dcae7946fa42c8a01127ec95491f8b118c4764069dc8074112c6"
Jul 16 11:29:16.725146 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:29:16.725084    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d84382d6bca2dcae7946fa42c8a01127ec95491f8b118c4764069dc8074112c6\": container with ID starting with d84382d6bca2dcae7946fa42c8a01127ec95491f8b118c4764069dc8074112c6 not found: ID does not exist" containerID="d84382d6bca2dcae7946fa42c8a01127ec95491f8b118c4764069dc8074112c6"
Jul 16 11:29:16.725146 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:29:16.725118    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d84382d6bca2dcae7946fa42c8a01127ec95491f8b118c4764069dc8074112c6"} err="failed to get container status \"d84382d6bca2dcae7946fa42c8a01127ec95491f8b118c4764069dc8074112c6\": rpc error: code = NotFound desc = could not find container \"d84382d6bca2dcae7946fa42c8a01127ec95491f8b118c4764069dc8074112c6\": container with ID starting with d84382d6bca2dcae7946fa42c8a01127ec95491f8b118c4764069dc8074112c6 not found: ID does not exist"
Jul 16 11:29:16.744807 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:29:16.744769    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-11372ed9-openshift-default-dc4cd7d99-tv64h"]
Jul 16 11:29:16.751131 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:29:16.751104    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-11372ed9-openshift-default-dc4cd7d99-tv64h"]
Jul 16 11:29:16.790188 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:29:16.790161    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-11372ed9-6c9bc5744f-8ncgj"
Jul 16 11:29:16.849091 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:29:16.849053    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-77gdk\" (UniqueName: \"kubernetes.io/projected/5f381ead-1bd5-4f97-a751-a232c8b07c83-kube-api-access-77gdk\") pod \"5f381ead-1bd5-4f97-a751-a232c8b07c83\" (UID: \"5f381ead-1bd5-4f97-a751-a232c8b07c83\") "
Jul 16 11:29:16.849249 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:29:16.849114    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f381ead-1bd5-4f97-a751-a232c8b07c83-openshift-service-ca\") pod \"5f381ead-1bd5-4f97-a751-a232c8b07c83\" (UID: \"5f381ead-1bd5-4f97-a751-a232c8b07c83\") "
Jul 16 11:29:16.849249 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:29:16.849149    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5f381ead-1bd5-4f97-a751-a232c8b07c83-maas-api-tls\") pod \"5f381ead-1bd5-4f97-a751-a232c8b07c83\" (UID: \"5f381ead-1bd5-4f97-a751-a232c8b07c83\") "
Jul 16 11:29:16.849503 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:29:16.849471    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5f381ead-1bd5-4f97-a751-a232c8b07c83-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "5f381ead-1bd5-4f97-a751-a232c8b07c83" (UID: "5f381ead-1bd5-4f97-a751-a232c8b07c83"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 11:29:16.851285 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:29:16.851261    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5f381ead-1bd5-4f97-a751-a232c8b07c83-kube-api-access-77gdk" (OuterVolumeSpecName: "kube-api-access-77gdk") pod "5f381ead-1bd5-4f97-a751-a232c8b07c83" (UID: "5f381ead-1bd5-4f97-a751-a232c8b07c83"). InnerVolumeSpecName "kube-api-access-77gdk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 11:29:16.851376 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:29:16.851288    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5f381ead-1bd5-4f97-a751-a232c8b07c83-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "5f381ead-1bd5-4f97-a751-a232c8b07c83" (UID: "5f381ead-1bd5-4f97-a751-a232c8b07c83"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 11:29:16.950129 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:29:16.950042    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-77gdk\" (UniqueName: \"kubernetes.io/projected/5f381ead-1bd5-4f97-a751-a232c8b07c83-kube-api-access-77gdk\") on node \"ip-10-0-129-28.ec2.internal\" DevicePath \"\""
Jul 16 11:29:16.950129 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:29:16.950077    2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f381ead-1bd5-4f97-a751-a232c8b07c83-openshift-service-ca\") on node \"ip-10-0-129-28.ec2.internal\" DevicePath \"\""
Jul 16 11:29:16.950129 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:29:16.950088    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5f381ead-1bd5-4f97-a751-a232c8b07c83-maas-api-tls\") on node \"ip-10-0-129-28.ec2.internal\" DevicePath \"\""
Jul 16 11:29:17.229063 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:29:17.228979    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="744e010e-fd4b-4535-ae6c-4e1e9680b073" path="/var/lib/kubelet/pods/744e010e-fd4b-4535-ae6c-4e1e9680b073/volumes"
Jul 16 11:29:17.724682 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:29:17.724656    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-11372ed9-6c9bc5744f-8ncgj"
Jul 16 11:29:17.725165 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:29:17.724655    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-11372ed9-6c9bc5744f-8ncgj" event={"ID":"5f381ead-1bd5-4f97-a751-a232c8b07c83","Type":"ContainerDied","Data":"1f9c3c144fa134a6761e170f79a8624ec8991f9dcb5e55dd60c611075fb81bf3"}
Jul 16 11:29:17.725165 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:29:17.724783    2583 scope.go:117] "RemoveContainer" containerID="643224b636707950a34009ed2a9715f1af5aef6d12c1a57526f743032844ab06"
Jul 16 11:29:17.743361 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:29:17.743330    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-11372ed9-6c9bc5744f-8ncgj"]
Jul 16 11:29:17.748160 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:29:17.748135    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-11372ed9-6c9bc5744f-8ncgj"]
Jul 16 11:29:19.229043 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:29:19.228997    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5f381ead-1bd5-4f97-a751-a232c8b07c83" path="/var/lib/kubelet/pods/5f381ead-1bd5-4f97-a751-a232c8b07c83/volumes"
Jul 16 11:30:00.145207 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:30:00.145165    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29736690-np49l"]
Jul 16 11:30:00.145721 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:30:00.145534    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f381ead-1bd5-4f97-a751-a232c8b07c83" containerName="maas-api"
Jul 16 11:30:00.145721 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:30:00.145547    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f381ead-1bd5-4f97-a751-a232c8b07c83" containerName="maas-api"
Jul 16 11:30:00.145721 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:30:00.145557    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="744e010e-fd4b-4535-ae6c-4e1e9680b073" containerName="istio-proxy"
Jul 16 11:30:00.145721 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:30:00.145563    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="744e010e-fd4b-4535-ae6c-4e1e9680b073" containerName="istio-proxy"
Jul 16 11:30:00.145721 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:30:00.145644    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="5f381ead-1bd5-4f97-a751-a232c8b07c83" containerName="maas-api"
Jul 16 11:30:00.145721 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:30:00.145656    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="744e010e-fd4b-4535-ae6c-4e1e9680b073" containerName="istio-proxy"
Jul 16 11:30:00.147546 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:30:00.147528    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29736690-np49l"
Jul 16 11:30:00.150264 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:30:00.150243    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-4cj8c\""
Jul 16 11:30:00.156981 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:30:00.156950    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29736690-np49l"]
Jul 16 11:30:00.159240 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:30:00.159219    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29736690-np49l"
Jul 16 11:30:00.314396 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:30:00.314368    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29736690-np49l"]
Jul 16 11:30:00.316930 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:30:00.316892    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf333fcd1_055b_4ee5_b0ea_39dd26a58d45.slice/crio-6124c5e834d89c1298ac2eb3f2d86b6c2bf4d3f63959015a937fdde5c8f3c365 WatchSource:0}: Error finding container 6124c5e834d89c1298ac2eb3f2d86b6c2bf4d3f63959015a937fdde5c8f3c365: Status 404 returned error can't find the container with id 6124c5e834d89c1298ac2eb3f2d86b6c2bf4d3f63959015a937fdde5c8f3c365
Jul 16 11:30:00.933084 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:30:00.933046    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29736690-np49l" event={"ID":"f333fcd1-055b-4ee5-b0ea-39dd26a58d45","Type":"ContainerStarted","Data":"7d109557c1d2b653063d9a721a6a8e007f51a83d43441666682c2460303cc08f"}
Jul 16 11:30:00.933260 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:30:00.933103    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29736690-np49l" event={"ID":"f333fcd1-055b-4ee5-b0ea-39dd26a58d45","Type":"ContainerStarted","Data":"6124c5e834d89c1298ac2eb3f2d86b6c2bf4d3f63959015a937fdde5c8f3c365"}
Jul 16 11:30:00.949731 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:30:00.949676    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29736690-np49l" podStartSLOduration=0.949660534 podStartE2EDuration="949.660534ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 11:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 11:30:00.947925659 +0000 UTC m=+1593.582147114" watchObservedRunningTime="2026-07-16 11:30:00.949660534 +0000 UTC m=+1593.583881987"
Jul 16 11:30:01.939299 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:30:01.939266    2583 generic.go:356] "Generic (PLEG): container finished" podID="f333fcd1-055b-4ee5-b0ea-39dd26a58d45" containerID="7d109557c1d2b653063d9a721a6a8e007f51a83d43441666682c2460303cc08f" exitCode=0
Jul 16 11:30:01.939729 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:30:01.939332    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29736690-np49l" event={"ID":"f333fcd1-055b-4ee5-b0ea-39dd26a58d45","Type":"ContainerDied","Data":"7d109557c1d2b653063d9a721a6a8e007f51a83d43441666682c2460303cc08f"}
Jul 16 11:30:03.079112 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:30:03.079085    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29736690-np49l"
Jul 16 11:30:03.949258 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:30:03.949228    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29736690-np49l"
Jul 16 11:30:03.949410 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:30:03.949269    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29736690-np49l" event={"ID":"f333fcd1-055b-4ee5-b0ea-39dd26a58d45","Type":"ContainerDied","Data":"6124c5e834d89c1298ac2eb3f2d86b6c2bf4d3f63959015a937fdde5c8f3c365"}
Jul 16 11:30:03.949410 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:30:03.949314    2583 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6124c5e834d89c1298ac2eb3f2d86b6c2bf4d3f63959015a937fdde5c8f3c365"
Jul 16 11:31:01.687234 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:31:01.687198    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-6c895ad5-openshift-default-585495bfb7-bsfwk"]
Jul 16 11:31:01.687856 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:31:01.687802    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f333fcd1-055b-4ee5-b0ea-39dd26a58d45" containerName="cleanup"
Jul 16 11:31:01.687856 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:31:01.687825    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="f333fcd1-055b-4ee5-b0ea-39dd26a58d45" containerName="cleanup"
Jul 16 11:31:01.688045 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:31:01.687949    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="f333fcd1-055b-4ee5-b0ea-39dd26a58d45" containerName="cleanup"
Jul 16 11:31:01.690767 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:31:01.690744    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-6c895ad5-openshift-default-585495bfb7-bsfwk"
Jul 16 11:31:01.693784 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:31:01.693763    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-6c895ad5-openshift-default-dockercfg-lzgsf\""
Jul 16 11:31:01.707979 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:31:01.707941    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-6c895ad5-openshift-default-585495bfb7-bsfwk"]
Jul 16 11:31:01.762541 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:31:01.762489    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/045d6242-b766-45ce-ae02-6fe5697e71bf-workload-certs\") pod \"e2e-mt-6c895ad5-openshift-default-585495bfb7-bsfwk\" (UID: \"045d6242-b766-45ce-ae02-6fe5697e71bf\") " pod="openshift-ingress/e2e-mt-6c895ad5-openshift-default-585495bfb7-bsfwk"
Jul 16 11:31:01.762764 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:31:01.762558    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/045d6242-b766-45ce-ae02-6fe5697e71bf-istio-data\") pod \"e2e-mt-6c895ad5-openshift-default-585495bfb7-bsfwk\" (UID: \"045d6242-b766-45ce-ae02-6fe5697e71bf\") " pod="openshift-ingress/e2e-mt-6c895ad5-openshift-default-585495bfb7-bsfwk"
Jul 16 11:31:01.762764 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:31:01.762587    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/045d6242-b766-45ce-ae02-6fe5697e71bf-istio-token\") pod \"e2e-mt-6c895ad5-openshift-default-585495bfb7-bsfwk\" (UID: \"045d6242-b766-45ce-ae02-6fe5697e71bf\") " pod="openshift-ingress/e2e-mt-6c895ad5-openshift-default-585495bfb7-bsfwk"
Jul 16 11:31:01.762764 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:31:01.762672    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/045d6242-b766-45ce-ae02-6fe5697e71bf-workload-socket\") pod \"e2e-mt-6c895ad5-openshift-default-585495bfb7-bsfwk\" (UID: \"045d6242-b766-45ce-ae02-6fe5697e71bf\") " pod="openshift-ingress/e2e-mt-6c895ad5-openshift-default-585495bfb7-bsfwk"
Jul 16 11:31:01.762764 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:31:01.762702    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/045d6242-b766-45ce-ae02-6fe5697e71bf-istiod-ca-cert\") pod \"e2e-mt-6c895ad5-openshift-default-585495bfb7-bsfwk\" (UID: \"045d6242-b766-45ce-ae02-6fe5697e71bf\") " pod="openshift-ingress/e2e-mt-6c895ad5-openshift-default-585495bfb7-bsfwk"
Jul 16 11:31:01.762764 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:31:01.762724    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/045d6242-b766-45ce-ae02-6fe5697e71bf-credential-socket\") pod \"e2e-mt-6c895ad5-openshift-default-585495bfb7-bsfwk\" (UID: \"045d6242-b766-45ce-ae02-6fe5697e71bf\") " pod="openshift-ingress/e2e-mt-6c895ad5-openshift-default-585495bfb7-bsfwk"
Jul 16 11:31:01.762764 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:31:01.762743    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rsh8q\" (UniqueName: \"kubernetes.io/projected/045d6242-b766-45ce-ae02-6fe5697e71bf-kube-api-access-rsh8q\") pod \"e2e-mt-6c895ad5-openshift-default-585495bfb7-bsfwk\" (UID: \"045d6242-b766-45ce-ae02-6fe5697e71bf\") " pod="openshift-ingress/e2e-mt-6c895ad5-openshift-default-585495bfb7-bsfwk"
Jul 16 11:31:01.762764 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:31:01.762761    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/045d6242-b766-45ce-ae02-6fe5697e71bf-istio-envoy\") pod \"e2e-mt-6c895ad5-openshift-default-585495bfb7-bsfwk\" (UID: \"045d6242-b766-45ce-ae02-6fe5697e71bf\") " pod="openshift-ingress/e2e-mt-6c895ad5-openshift-default-585495bfb7-bsfwk"
Jul 16 11:31:01.763007 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:31:01.762856    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/045d6242-b766-45ce-ae02-6fe5697e71bf-istio-podinfo\") pod \"e2e-mt-6c895ad5-openshift-default-585495bfb7-bsfwk\" (UID: \"045d6242-b766-45ce-ae02-6fe5697e71bf\") " pod="openshift-ingress/e2e-mt-6c895ad5-openshift-default-585495bfb7-bsfwk"
Jul 16 11:31:01.863997 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:31:01.863953    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/045d6242-b766-45ce-ae02-6fe5697e71bf-workload-socket\") pod \"e2e-mt-6c895ad5-openshift-default-585495bfb7-bsfwk\" (UID: \"045d6242-b766-45ce-ae02-6fe5697e71bf\") " pod="openshift-ingress/e2e-mt-6c895ad5-openshift-default-585495bfb7-bsfwk"
Jul 16 11:31:01.863997 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:31:01.863996    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/045d6242-b766-45ce-ae02-6fe5697e71bf-istiod-ca-cert\") pod \"e2e-mt-6c895ad5-openshift-default-585495bfb7-bsfwk\" (UID: \"045d6242-b766-45ce-ae02-6fe5697e71bf\") " pod="openshift-ingress/e2e-mt-6c895ad5-openshift-default-585495bfb7-bsfwk"
Jul 16 11:31:01.864274 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:31:01.864015    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/045d6242-b766-45ce-ae02-6fe5697e71bf-credential-socket\") pod \"e2e-mt-6c895ad5-openshift-default-585495bfb7-bsfwk\" (UID: \"045d6242-b766-45ce-ae02-6fe5697e71bf\") " pod="openshift-ingress/e2e-mt-6c895ad5-openshift-default-585495bfb7-bsfwk"
Jul 16 11:31:01.864274 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:31:01.864034    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rsh8q\" (UniqueName: \"kubernetes.io/projected/045d6242-b766-45ce-ae02-6fe5697e71bf-kube-api-access-rsh8q\") pod \"e2e-mt-6c895ad5-openshift-default-585495bfb7-bsfwk\" (UID: \"045d6242-b766-45ce-ae02-6fe5697e71bf\") " pod="openshift-ingress/e2e-mt-6c895ad5-openshift-default-585495bfb7-bsfwk"
Jul 16 11:31:01.864274 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:31:01.864142    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/045d6242-b766-45ce-ae02-6fe5697e71bf-istio-envoy\") pod \"e2e-mt-6c895ad5-openshift-default-585495bfb7-bsfwk\" (UID: \"045d6242-b766-45ce-ae02-6fe5697e71bf\") " pod="openshift-ingress/e2e-mt-6c895ad5-openshift-default-585495bfb7-bsfwk"
Jul 16 11:31:01.864274 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:31:01.864199    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/045d6242-b766-45ce-ae02-6fe5697e71bf-istio-podinfo\") pod \"e2e-mt-6c895ad5-openshift-default-585495bfb7-bsfwk\" (UID: \"045d6242-b766-45ce-ae02-6fe5697e71bf\") " pod="openshift-ingress/e2e-mt-6c895ad5-openshift-default-585495bfb7-bsfwk"
Jul 16 11:31:01.864274 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:31:01.864233    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/045d6242-b766-45ce-ae02-6fe5697e71bf-workload-certs\") pod \"e2e-mt-6c895ad5-openshift-default-585495bfb7-bsfwk\" (UID: \"045d6242-b766-45ce-ae02-6fe5697e71bf\") " pod="openshift-ingress/e2e-mt-6c895ad5-openshift-default-585495bfb7-bsfwk"
Jul 16 11:31:01.864537 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:31:01.864297    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/045d6242-b766-45ce-ae02-6fe5697e71bf-istio-data\") pod \"e2e-mt-6c895ad5-openshift-default-585495bfb7-bsfwk\" (UID: \"045d6242-b766-45ce-ae02-6fe5697e71bf\") " pod="openshift-ingress/e2e-mt-6c895ad5-openshift-default-585495bfb7-bsfwk"
Jul 16 11:31:01.864537 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:31:01.864507    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/045d6242-b766-45ce-ae02-6fe5697e71bf-credential-socket\") pod \"e2e-mt-6c895ad5-openshift-default-585495bfb7-bsfwk\" (UID: \"045d6242-b766-45ce-ae02-6fe5697e71bf\") " pod="openshift-ingress/e2e-mt-6c895ad5-openshift-default-585495bfb7-bsfwk"
Jul 16 11:31:01.864679 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:31:01.864623    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/045d6242-b766-45ce-ae02-6fe5697e71bf-workload-certs\") pod \"e2e-mt-6c895ad5-openshift-default-585495bfb7-bsfwk\" (UID: \"045d6242-b766-45ce-ae02-6fe5697e71bf\") " pod="openshift-ingress/e2e-mt-6c895ad5-openshift-default-585495bfb7-bsfwk"
Jul 16 11:31:01.864679 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:31:01.864631    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/045d6242-b766-45ce-ae02-6fe5697e71bf-istio-token\") pod \"e2e-mt-6c895ad5-openshift-default-585495bfb7-bsfwk\" (UID: \"045d6242-b766-45ce-ae02-6fe5697e71bf\") " pod="openshift-ingress/e2e-mt-6c895ad5-openshift-default-585495bfb7-bsfwk"
Jul 16 11:31:01.864824 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:31:01.864797    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/045d6242-b766-45ce-ae02-6fe5697e71bf-istio-data\") pod \"e2e-mt-6c895ad5-openshift-default-585495bfb7-bsfwk\" (UID: \"045d6242-b766-45ce-ae02-6fe5697e71bf\") " pod="openshift-ingress/e2e-mt-6c895ad5-openshift-default-585495bfb7-bsfwk"
Jul 16 11:31:01.864886 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:31:01.864823    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/045d6242-b766-45ce-ae02-6fe5697e71bf-workload-socket\") pod \"e2e-mt-6c895ad5-openshift-default-585495bfb7-bsfwk\" (UID: \"045d6242-b766-45ce-ae02-6fe5697e71bf\") " pod="openshift-ingress/e2e-mt-6c895ad5-openshift-default-585495bfb7-bsfwk"
Jul 16 11:31:01.864959 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:31:01.864920    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/045d6242-b766-45ce-ae02-6fe5697e71bf-istiod-ca-cert\") pod \"e2e-mt-6c895ad5-openshift-default-585495bfb7-bsfwk\" (UID: \"045d6242-b766-45ce-ae02-6fe5697e71bf\") " pod="openshift-ingress/e2e-mt-6c895ad5-openshift-default-585495bfb7-bsfwk"
Jul 16 11:31:01.866820 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:31:01.866795    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/045d6242-b766-45ce-ae02-6fe5697e71bf-istio-envoy\") pod \"e2e-mt-6c895ad5-openshift-default-585495bfb7-bsfwk\" (UID: \"045d6242-b766-45ce-ae02-6fe5697e71bf\") " pod="openshift-ingress/e2e-mt-6c895ad5-openshift-default-585495bfb7-bsfwk"
Jul 16 11:31:01.866946 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:31:01.866925    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/045d6242-b766-45ce-ae02-6fe5697e71bf-istio-podinfo\") pod \"e2e-mt-6c895ad5-openshift-default-585495bfb7-bsfwk\" (UID: \"045d6242-b766-45ce-ae02-6fe5697e71bf\") " pod="openshift-ingress/e2e-mt-6c895ad5-openshift-default-585495bfb7-bsfwk"
Jul 16 11:31:01.874296 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:31:01.874256    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rsh8q\" (UniqueName: \"kubernetes.io/projected/045d6242-b766-45ce-ae02-6fe5697e71bf-kube-api-access-rsh8q\") pod \"e2e-mt-6c895ad5-openshift-default-585495bfb7-bsfwk\" (UID: \"045d6242-b766-45ce-ae02-6fe5697e71bf\") " pod="openshift-ingress/e2e-mt-6c895ad5-openshift-default-585495bfb7-bsfwk"
Jul 16 11:31:01.874436 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:31:01.874263    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/045d6242-b766-45ce-ae02-6fe5697e71bf-istio-token\") pod \"e2e-mt-6c895ad5-openshift-default-585495bfb7-bsfwk\" (UID: \"045d6242-b766-45ce-ae02-6fe5697e71bf\") " pod="openshift-ingress/e2e-mt-6c895ad5-openshift-default-585495bfb7-bsfwk"
Jul 16 11:31:02.001761 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:31:02.001659    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-6c895ad5-openshift-default-585495bfb7-bsfwk"
Jul 16 11:31:02.157968 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:31:02.157941    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-6c895ad5-openshift-default-585495bfb7-bsfwk"]
Jul 16 11:31:02.160022 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:31:02.159994    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod045d6242_b766_45ce_ae02_6fe5697e71bf.slice/crio-03e62c44cc579c2c7baae43fda547f4e2aafbb3789ed94ad0b8eb5389db97339 WatchSource:0}: Error finding container 03e62c44cc579c2c7baae43fda547f4e2aafbb3789ed94ad0b8eb5389db97339: Status 404 returned error can't find the container with id 03e62c44cc579c2c7baae43fda547f4e2aafbb3789ed94ad0b8eb5389db97339
Jul 16 11:31:02.162307 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:31:02.162276    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 11:31:02.162371 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:31:02.162340    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 11:31:02.162371 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:31:02.162366    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 11:31:02.223435 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:31:02.223409    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-6c895ad5-openshift-default-585495bfb7-bsfwk" event={"ID":"045d6242-b766-45ce-ae02-6fe5697e71bf","Type":"ContainerStarted","Data":"03e62c44cc579c2c7baae43fda547f4e2aafbb3789ed94ad0b8eb5389db97339"}
Jul 16 11:31:03.229333 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:31:03.229293    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-6c895ad5-openshift-default-585495bfb7-bsfwk" event={"ID":"045d6242-b766-45ce-ae02-6fe5697e71bf","Type":"ContainerStarted","Data":"a7df3902deb1884e211bfcb997eecb3782a4f6b08175ee4abd469b8ac15b0400"}
Jul 16 11:31:03.255901 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:31:03.255735    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-6c895ad5-openshift-default-585495bfb7-bsfwk" podStartSLOduration=2.255715427 podStartE2EDuration="2.255715427s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 11:31:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 11:31:03.255459628 +0000 UTC m=+1655.889681109" watchObservedRunningTime="2026-07-16 11:31:03.255715427 +0000 UTC m=+1655.889936882"
Jul 16 11:31:04.002314 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:31:04.002271    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-6c895ad5-openshift-default-585495bfb7-bsfwk"
Jul 16 11:31:04.007418 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:31:04.007390    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-6c895ad5-openshift-default-585495bfb7-bsfwk"
Jul 16 11:31:04.234013 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:31:04.233981    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-6c895ad5-openshift-default-585495bfb7-bsfwk"
Jul 16 11:31:04.234938 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:31:04.234917    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-6c895ad5-openshift-default-585495bfb7-bsfwk"
Jul 16 11:31:04.283457 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:31:04.283361    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-6c895ad5-5cb9cdf87b-sdp66"]
Jul 16 11:31:04.287731 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:31:04.287693    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6c895ad5-5cb9cdf87b-sdp66"
Jul 16 11:31:04.290167 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:31:04.290140    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-6c895ad5-serving-cert\""
Jul 16 11:31:04.297634 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:31:04.297590    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-6c895ad5-5cb9cdf87b-sdp66"]
Jul 16 11:31:04.387466 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:31:04.387428    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5fa9839e-d970-47d8-88a6-1c041f8a7eb5-openshift-service-ca\") pod \"maas-api-e2e-mt-6c895ad5-5cb9cdf87b-sdp66\" (UID: \"5fa9839e-d970-47d8-88a6-1c041f8a7eb5\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6c895ad5-5cb9cdf87b-sdp66"
Jul 16 11:31:04.387693 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:31:04.387483    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5fa9839e-d970-47d8-88a6-1c041f8a7eb5-maas-api-tls\") pod \"maas-api-e2e-mt-6c895ad5-5cb9cdf87b-sdp66\" (UID: \"5fa9839e-d970-47d8-88a6-1c041f8a7eb5\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6c895ad5-5cb9cdf87b-sdp66"
Jul 16 11:31:04.387693 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:31:04.387561    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-txfkj\" (UniqueName: \"kubernetes.io/projected/5fa9839e-d970-47d8-88a6-1c041f8a7eb5-kube-api-access-txfkj\") pod \"maas-api-e2e-mt-6c895ad5-5cb9cdf87b-sdp66\" (UID: \"5fa9839e-d970-47d8-88a6-1c041f8a7eb5\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6c895ad5-5cb9cdf87b-sdp66"
Jul 16 11:31:04.488978 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:31:04.488929    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5fa9839e-d970-47d8-88a6-1c041f8a7eb5-openshift-service-ca\") pod \"maas-api-e2e-mt-6c895ad5-5cb9cdf87b-sdp66\" (UID: \"5fa9839e-d970-47d8-88a6-1c041f8a7eb5\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6c895ad5-5cb9cdf87b-sdp66"
Jul 16 11:31:04.488978 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:31:04.488996    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5fa9839e-d970-47d8-88a6-1c041f8a7eb5-maas-api-tls\") pod \"maas-api-e2e-mt-6c895ad5-5cb9cdf87b-sdp66\" (UID: \"5fa9839e-d970-47d8-88a6-1c041f8a7eb5\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6c895ad5-5cb9cdf87b-sdp66"
Jul 16 11:31:04.489273 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:31:04.489072    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-txfkj\" (UniqueName: \"kubernetes.io/projected/5fa9839e-d970-47d8-88a6-1c041f8a7eb5-kube-api-access-txfkj\") pod \"maas-api-e2e-mt-6c895ad5-5cb9cdf87b-sdp66\" (UID: \"5fa9839e-d970-47d8-88a6-1c041f8a7eb5\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6c895ad5-5cb9cdf87b-sdp66"
Jul 16 11:31:04.489273 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:31:04.489165    2583 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-6c895ad5-serving-cert: secret "maas-api-e2e-mt-6c895ad5-serving-cert" not found
Jul 16 11:31:04.489273 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:31:04.489243    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5fa9839e-d970-47d8-88a6-1c041f8a7eb5-maas-api-tls podName:5fa9839e-d970-47d8-88a6-1c041f8a7eb5 nodeName:}" failed. No retries permitted until 2026-07-16 11:31:04.989222111 +0000 UTC m=+1657.623443543 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/5fa9839e-d970-47d8-88a6-1c041f8a7eb5-maas-api-tls") pod "maas-api-e2e-mt-6c895ad5-5cb9cdf87b-sdp66" (UID: "5fa9839e-d970-47d8-88a6-1c041f8a7eb5") : secret "maas-api-e2e-mt-6c895ad5-serving-cert" not found
Jul 16 11:31:04.489646 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:31:04.489591    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5fa9839e-d970-47d8-88a6-1c041f8a7eb5-openshift-service-ca\") pod \"maas-api-e2e-mt-6c895ad5-5cb9cdf87b-sdp66\" (UID: \"5fa9839e-d970-47d8-88a6-1c041f8a7eb5\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6c895ad5-5cb9cdf87b-sdp66"
Jul 16 11:31:04.498732 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:31:04.498704    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-txfkj\" (UniqueName: \"kubernetes.io/projected/5fa9839e-d970-47d8-88a6-1c041f8a7eb5-kube-api-access-txfkj\") pod \"maas-api-e2e-mt-6c895ad5-5cb9cdf87b-sdp66\" (UID: \"5fa9839e-d970-47d8-88a6-1c041f8a7eb5\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6c895ad5-5cb9cdf87b-sdp66"
Jul 16 11:31:04.994012 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:31:04.993973    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5fa9839e-d970-47d8-88a6-1c041f8a7eb5-maas-api-tls\") pod \"maas-api-e2e-mt-6c895ad5-5cb9cdf87b-sdp66\" (UID: \"5fa9839e-d970-47d8-88a6-1c041f8a7eb5\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6c895ad5-5cb9cdf87b-sdp66"
Jul 16 11:31:04.994207 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:31:04.994140    2583 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-6c895ad5-serving-cert: secret "maas-api-e2e-mt-6c895ad5-serving-cert" not found
Jul 16 11:31:04.994207 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:31:04.994206    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5fa9839e-d970-47d8-88a6-1c041f8a7eb5-maas-api-tls podName:5fa9839e-d970-47d8-88a6-1c041f8a7eb5 nodeName:}" failed. No retries permitted until 2026-07-16 11:31:05.994189402 +0000 UTC m=+1658.628410834 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/5fa9839e-d970-47d8-88a6-1c041f8a7eb5-maas-api-tls") pod "maas-api-e2e-mt-6c895ad5-5cb9cdf87b-sdp66" (UID: "5fa9839e-d970-47d8-88a6-1c041f8a7eb5") : secret "maas-api-e2e-mt-6c895ad5-serving-cert" not found
Jul 16 11:31:06.003657 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:31:06.003586    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5fa9839e-d970-47d8-88a6-1c041f8a7eb5-maas-api-tls\") pod \"maas-api-e2e-mt-6c895ad5-5cb9cdf87b-sdp66\" (UID: \"5fa9839e-d970-47d8-88a6-1c041f8a7eb5\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6c895ad5-5cb9cdf87b-sdp66"
Jul 16 11:31:06.006090 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:31:06.006056    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5fa9839e-d970-47d8-88a6-1c041f8a7eb5-maas-api-tls\") pod \"maas-api-e2e-mt-6c895ad5-5cb9cdf87b-sdp66\" (UID: \"5fa9839e-d970-47d8-88a6-1c041f8a7eb5\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6c895ad5-5cb9cdf87b-sdp66"
Jul 16 11:31:06.103219 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:31:06.103172    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6c895ad5-5cb9cdf87b-sdp66"
Jul 16 11:31:06.247536 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:31:06.247511    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-6c895ad5-5cb9cdf87b-sdp66"]
Jul 16 11:31:06.249526 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:31:06.249487    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5fa9839e_d970_47d8_88a6_1c041f8a7eb5.slice/crio-755562e330df4ab8763d4ede19a39b48e8ad61beabc23a4dd96249eee8e79bfd WatchSource:0}: Error finding container 755562e330df4ab8763d4ede19a39b48e8ad61beabc23a4dd96249eee8e79bfd: Status 404 returned error can't find the container with id 755562e330df4ab8763d4ede19a39b48e8ad61beabc23a4dd96249eee8e79bfd
Jul 16 11:31:07.252579 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:31:07.252539    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6c895ad5-5cb9cdf87b-sdp66" event={"ID":"5fa9839e-d970-47d8-88a6-1c041f8a7eb5","Type":"ContainerStarted","Data":"a1b91b4c3aca9614b68ca486c56f1c9028e3cf327e85de53b2bf6f92310ee0a3"}
Jul 16 11:31:07.253082 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:31:07.252623    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6c895ad5-5cb9cdf87b-sdp66" event={"ID":"5fa9839e-d970-47d8-88a6-1c041f8a7eb5","Type":"ContainerStarted","Data":"755562e330df4ab8763d4ede19a39b48e8ad61beabc23a4dd96249eee8e79bfd"}
Jul 16 11:31:07.253082 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:31:07.252718    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6c895ad5-5cb9cdf87b-sdp66"
Jul 16 11:31:07.272397 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:31:07.272334    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6c895ad5-5cb9cdf87b-sdp66" podStartSLOduration=2.8222598100000003 podStartE2EDuration="3.272316939s" totalImagesPullingTime="450.057129ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 11:31:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 11:31:07.271635456 +0000 UTC m=+1659.905856937" watchObservedRunningTime="2026-07-16 11:31:07.272316939 +0000 UTC m=+1659.906538392"
Jul 16 11:31:13.263127 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:31:13.263095    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6c895ad5-5cb9cdf87b-sdp66"
Jul 16 11:31:18.305809 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:31:18.305773    2583 generic.go:356] "Generic (PLEG): container finished" podID="045d6242-b766-45ce-ae02-6fe5697e71bf" containerID="a7df3902deb1884e211bfcb997eecb3782a4f6b08175ee4abd469b8ac15b0400" exitCode=0
Jul 16 11:31:18.306259 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:31:18.305849    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-6c895ad5-openshift-default-585495bfb7-bsfwk" event={"ID":"045d6242-b766-45ce-ae02-6fe5697e71bf","Type":"ContainerDied","Data":"a7df3902deb1884e211bfcb997eecb3782a4f6b08175ee4abd469b8ac15b0400"}
Jul 16 11:31:18.306316 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:31:18.306265    2583 scope.go:117] "RemoveContainer" containerID="a7df3902deb1884e211bfcb997eecb3782a4f6b08175ee4abd469b8ac15b0400"
Jul 16 11:31:18.307310 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:31:18.307284    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 11:31:18.307370 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:31:18.307352    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 11:31:18.307407 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:31:18.307382    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 11:31:19.242726 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:31:19.242624    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-6c895ad5-openshift-default-585495bfb7-bsfwk"
Jul 16 11:31:19.312678 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:31:19.312631    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-6c895ad5-openshift-default-585495bfb7-bsfwk" event={"ID":"045d6242-b766-45ce-ae02-6fe5697e71bf","Type":"ContainerStarted","Data":"6e3034359f05bb1886e9fcab4686210277b2d6fa97ad66cbb0b31630c60356f9"}
Jul 16 11:31:20.002420 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:31:20.002383    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-6c895ad5-openshift-default-585495bfb7-bsfwk"
Jul 16 11:31:20.003657 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:31:20.003625    2583 patch_prober.go:28] interesting pod/e2e-mt-6c895ad5-openshift-default-585495bfb7-bsfwk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body=
Jul 16 11:31:20.003780 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:31:20.003684    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-6c895ad5-openshift-default-585495bfb7-bsfwk" podUID="045d6242-b766-45ce-ae02-6fe5697e71bf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused"
Jul 16 11:31:21.002766 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:31:21.002725    2583 patch_prober.go:28] interesting pod/e2e-mt-6c895ad5-openshift-default-585495bfb7-bsfwk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body=
Jul 16 11:31:21.003171 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:31:21.002783    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-6c895ad5-openshift-default-585495bfb7-bsfwk" podUID="045d6242-b766-45ce-ae02-6fe5697e71bf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused"
Jul 16 11:31:22.002468 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:31:22.002427    2583 patch_prober.go:28] interesting pod/e2e-mt-6c895ad5-openshift-default-585495bfb7-bsfwk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body=
Jul 16 11:31:22.002681 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:31:22.002502    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-6c895ad5-openshift-default-585495bfb7-bsfwk" podUID="045d6242-b766-45ce-ae02-6fe5697e71bf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused"
Jul 16 11:31:23.002523 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:31:23.002485    2583 patch_prober.go:28] interesting pod/e2e-mt-6c895ad5-openshift-default-585495bfb7-bsfwk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body=
Jul 16 11:31:23.002933 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:31:23.002560    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-6c895ad5-openshift-default-585495bfb7-bsfwk" podUID="045d6242-b766-45ce-ae02-6fe5697e71bf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused"
Jul 16 11:31:23.040478 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:31:23.040063    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-6c895ad5-5cb9cdf87b-sdp66"]
Jul 16 11:31:23.040684 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:31:23.040504    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6c895ad5-5cb9cdf87b-sdp66" podUID="5fa9839e-d970-47d8-88a6-1c041f8a7eb5" containerName="maas-api" containerID="cri-o://a1b91b4c3aca9614b68ca486c56f1c9028e3cf327e85de53b2bf6f92310ee0a3" gracePeriod=30
Jul 16 11:31:23.294294 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:31:23.294266    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6c895ad5-5cb9cdf87b-sdp66"
Jul 16 11:31:23.339607 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:31:23.339557    2583 generic.go:356] "Generic (PLEG): container finished" podID="5fa9839e-d970-47d8-88a6-1c041f8a7eb5" containerID="a1b91b4c3aca9614b68ca486c56f1c9028e3cf327e85de53b2bf6f92310ee0a3" exitCode=0
Jul 16 11:31:23.339779 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:31:23.339637    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6c895ad5-5cb9cdf87b-sdp66"
Jul 16 11:31:23.339779 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:31:23.339646    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6c895ad5-5cb9cdf87b-sdp66" event={"ID":"5fa9839e-d970-47d8-88a6-1c041f8a7eb5","Type":"ContainerDied","Data":"a1b91b4c3aca9614b68ca486c56f1c9028e3cf327e85de53b2bf6f92310ee0a3"}
Jul 16 11:31:23.339779 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:31:23.339699    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6c895ad5-5cb9cdf87b-sdp66" event={"ID":"5fa9839e-d970-47d8-88a6-1c041f8a7eb5","Type":"ContainerDied","Data":"755562e330df4ab8763d4ede19a39b48e8ad61beabc23a4dd96249eee8e79bfd"}
Jul 16 11:31:23.339779 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:31:23.339722    2583 scope.go:117] "RemoveContainer" containerID="a1b91b4c3aca9614b68ca486c56f1c9028e3cf327e85de53b2bf6f92310ee0a3"
Jul 16 11:31:23.348910 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:31:23.348883    2583 scope.go:117] "RemoveContainer" containerID="a1b91b4c3aca9614b68ca486c56f1c9028e3cf327e85de53b2bf6f92310ee0a3"
Jul 16 11:31:23.349255 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:31:23.349226    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a1b91b4c3aca9614b68ca486c56f1c9028e3cf327e85de53b2bf6f92310ee0a3\": container with ID starting with a1b91b4c3aca9614b68ca486c56f1c9028e3cf327e85de53b2bf6f92310ee0a3 not found: ID does not exist" containerID="a1b91b4c3aca9614b68ca486c56f1c9028e3cf327e85de53b2bf6f92310ee0a3"
Jul 16 11:31:23.349388 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:31:23.349259    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a1b91b4c3aca9614b68ca486c56f1c9028e3cf327e85de53b2bf6f92310ee0a3"} err="failed to get container status \"a1b91b4c3aca9614b68ca486c56f1c9028e3cf327e85de53b2bf6f92310ee0a3\": rpc error: code = NotFound desc = could not find container \"a1b91b4c3aca9614b68ca486c56f1c9028e3cf327e85de53b2bf6f92310ee0a3\": container with ID starting with a1b91b4c3aca9614b68ca486c56f1c9028e3cf327e85de53b2bf6f92310ee0a3 not found: ID does not exist"
Jul 16 11:31:23.380652 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:31:23.380591    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5fa9839e-d970-47d8-88a6-1c041f8a7eb5-maas-api-tls\") pod \"5fa9839e-d970-47d8-88a6-1c041f8a7eb5\" (UID: \"5fa9839e-d970-47d8-88a6-1c041f8a7eb5\") "
Jul 16 11:31:23.380831 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:31:23.380721    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-txfkj\" (UniqueName: \"kubernetes.io/projected/5fa9839e-d970-47d8-88a6-1c041f8a7eb5-kube-api-access-txfkj\") pod \"5fa9839e-d970-47d8-88a6-1c041f8a7eb5\" (UID: \"5fa9839e-d970-47d8-88a6-1c041f8a7eb5\") "
Jul 16 11:31:23.380831 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:31:23.380795    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5fa9839e-d970-47d8-88a6-1c041f8a7eb5-openshift-service-ca\") pod \"5fa9839e-d970-47d8-88a6-1c041f8a7eb5\" (UID: \"5fa9839e-d970-47d8-88a6-1c041f8a7eb5\") "
Jul 16 11:31:23.381132 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:31:23.381104    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5fa9839e-d970-47d8-88a6-1c041f8a7eb5-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "5fa9839e-d970-47d8-88a6-1c041f8a7eb5" (UID: "5fa9839e-d970-47d8-88a6-1c041f8a7eb5"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 11:31:23.382816 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:31:23.382791    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5fa9839e-d970-47d8-88a6-1c041f8a7eb5-kube-api-access-txfkj" (OuterVolumeSpecName: "kube-api-access-txfkj") pod "5fa9839e-d970-47d8-88a6-1c041f8a7eb5" (UID: "5fa9839e-d970-47d8-88a6-1c041f8a7eb5"). InnerVolumeSpecName "kube-api-access-txfkj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 11:31:23.382919 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:31:23.382843    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5fa9839e-d970-47d8-88a6-1c041f8a7eb5-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "5fa9839e-d970-47d8-88a6-1c041f8a7eb5" (UID: "5fa9839e-d970-47d8-88a6-1c041f8a7eb5"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 11:31:23.482229 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:31:23.482187    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-txfkj\" (UniqueName: \"kubernetes.io/projected/5fa9839e-d970-47d8-88a6-1c041f8a7eb5-kube-api-access-txfkj\") on node \"ip-10-0-129-28.ec2.internal\" DevicePath \"\""
Jul 16 11:31:23.482229 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:31:23.482223    2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5fa9839e-d970-47d8-88a6-1c041f8a7eb5-openshift-service-ca\") on node \"ip-10-0-129-28.ec2.internal\" DevicePath \"\""
Jul 16 11:31:23.482229 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:31:23.482235    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5fa9839e-d970-47d8-88a6-1c041f8a7eb5-maas-api-tls\") on node \"ip-10-0-129-28.ec2.internal\" DevicePath \"\""
Jul 16 11:31:23.663399 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:31:23.663357    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-6c895ad5-5cb9cdf87b-sdp66"]
Jul 16 11:31:23.669384 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:31:23.669333    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-6c895ad5-5cb9cdf87b-sdp66"]
Jul 16 11:31:24.006400 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:31:24.006321    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-6c895ad5-openshift-default-585495bfb7-bsfwk"
Jul 16 11:31:24.006808 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:31:24.006729    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-6c895ad5-openshift-default-585495bfb7-bsfwk"
Jul 16 11:31:24.007445 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:31:24.007427    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-6c895ad5-openshift-default-585495bfb7-bsfwk"
Jul 16 11:31:25.236931 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:31:25.236886    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5fa9839e-d970-47d8-88a6-1c041f8a7eb5" path="/var/lib/kubelet/pods/5fa9839e-d970-47d8-88a6-1c041f8a7eb5/volumes"
Jul 16 11:31:26.258254 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:31:26.258202    2583 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6c895ad5-5cb9cdf87b-sdp66" podUID="5fa9839e-d970-47d8-88a6-1c041f8a7eb5" containerName="maas-api" probeResult="failure" output="Get \"https://10.132.0.48:8443/health\": context deadline exceeded"
Jul 16 11:31:30.926113 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:31:30.926076    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-6c895ad5-openshift-default-585495bfb7-bsfwk"]
Jul 16 11:31:30.926567 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:31:30.926279    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-6c895ad5-openshift-default-585495bfb7-bsfwk" podUID="045d6242-b766-45ce-ae02-6fe5697e71bf" containerName="istio-proxy" containerID="cri-o://6e3034359f05bb1886e9fcab4686210277b2d6fa97ad66cbb0b31630c60356f9" gracePeriod=30
Jul 16 11:31:36.179510 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:31:36.179481    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-6c895ad5-openshift-default-585495bfb7-bsfwk"
Jul 16 11:31:36.300920 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:31:36.300814    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/045d6242-b766-45ce-ae02-6fe5697e71bf-istiod-ca-cert\") pod \"045d6242-b766-45ce-ae02-6fe5697e71bf\" (UID: \"045d6242-b766-45ce-ae02-6fe5697e71bf\") "
Jul 16 11:31:36.300920 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:31:36.300854    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/045d6242-b766-45ce-ae02-6fe5697e71bf-istio-envoy\") pod \"045d6242-b766-45ce-ae02-6fe5697e71bf\" (UID: \"045d6242-b766-45ce-ae02-6fe5697e71bf\") "
Jul 16 11:31:36.300920 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:31:36.300898    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/045d6242-b766-45ce-ae02-6fe5697e71bf-workload-socket\") pod \"045d6242-b766-45ce-ae02-6fe5697e71bf\" (UID: \"045d6242-b766-45ce-ae02-6fe5697e71bf\") "
Jul 16 11:31:36.301228 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:31:36.301064    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/045d6242-b766-45ce-ae02-6fe5697e71bf-istio-data\") pod \"045d6242-b766-45ce-ae02-6fe5697e71bf\" (UID: \"045d6242-b766-45ce-ae02-6fe5697e71bf\") "
Jul 16 11:31:36.301228 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:31:36.301110    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/045d6242-b766-45ce-ae02-6fe5697e71bf-workload-certs\") pod \"045d6242-b766-45ce-ae02-6fe5697e71bf\" (UID: \"045d6242-b766-45ce-ae02-6fe5697e71bf\") "
Jul 16 11:31:36.301228 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:31:36.301132    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/045d6242-b766-45ce-ae02-6fe5697e71bf-istio-podinfo\") pod \"045d6242-b766-45ce-ae02-6fe5697e71bf\" (UID: \"045d6242-b766-45ce-ae02-6fe5697e71bf\") "
Jul 16 11:31:36.301228 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:31:36.301165    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/045d6242-b766-45ce-ae02-6fe5697e71bf-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "045d6242-b766-45ce-ae02-6fe5697e71bf" (UID: "045d6242-b766-45ce-ae02-6fe5697e71bf"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 11:31:36.301228 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:31:36.301213    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/045d6242-b766-45ce-ae02-6fe5697e71bf-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "045d6242-b766-45ce-ae02-6fe5697e71bf" (UID: "045d6242-b766-45ce-ae02-6fe5697e71bf"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 11:31:36.301490 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:31:36.301234    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/045d6242-b766-45ce-ae02-6fe5697e71bf-istio-token\") pod \"045d6242-b766-45ce-ae02-6fe5697e71bf\" (UID: \"045d6242-b766-45ce-ae02-6fe5697e71bf\") "
Jul 16 11:31:36.301490 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:31:36.301263    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rsh8q\" (UniqueName: \"kubernetes.io/projected/045d6242-b766-45ce-ae02-6fe5697e71bf-kube-api-access-rsh8q\") pod \"045d6242-b766-45ce-ae02-6fe5697e71bf\" (UID: \"045d6242-b766-45ce-ae02-6fe5697e71bf\") "
Jul 16 11:31:36.301490 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:31:36.301292    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/045d6242-b766-45ce-ae02-6fe5697e71bf-credential-socket\") pod \"045d6242-b766-45ce-ae02-6fe5697e71bf\" (UID: \"045d6242-b766-45ce-ae02-6fe5697e71bf\") "
Jul 16 11:31:36.301490 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:31:36.301344    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/045d6242-b766-45ce-ae02-6fe5697e71bf-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "045d6242-b766-45ce-ae02-6fe5697e71bf" (UID: "045d6242-b766-45ce-ae02-6fe5697e71bf"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 11:31:36.301730 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:31:36.301621    2583 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/045d6242-b766-45ce-ae02-6fe5697e71bf-istiod-ca-cert\") on node \"ip-10-0-129-28.ec2.internal\" DevicePath \"\""
Jul 16 11:31:36.301730 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:31:36.301643    2583 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/045d6242-b766-45ce-ae02-6fe5697e71bf-workload-socket\") on node \"ip-10-0-129-28.ec2.internal\" DevicePath \"\""
Jul 16 11:31:36.301730 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:31:36.301643    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/045d6242-b766-45ce-ae02-6fe5697e71bf-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "045d6242-b766-45ce-ae02-6fe5697e71bf" (UID: "045d6242-b766-45ce-ae02-6fe5697e71bf"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 11:31:36.301730 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:31:36.301657    2583 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/045d6242-b766-45ce-ae02-6fe5697e71bf-workload-certs\") on node \"ip-10-0-129-28.ec2.internal\" DevicePath \"\""
Jul 16 11:31:36.302569 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:31:36.302518    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/045d6242-b766-45ce-ae02-6fe5697e71bf-istio-data" (OuterVolumeSpecName: "istio-data") pod "045d6242-b766-45ce-ae02-6fe5697e71bf" (UID: "045d6242-b766-45ce-ae02-6fe5697e71bf"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 11:31:36.310864 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:31:36.310777    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/045d6242-b766-45ce-ae02-6fe5697e71bf-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "045d6242-b766-45ce-ae02-6fe5697e71bf" (UID: "045d6242-b766-45ce-ae02-6fe5697e71bf"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 11:31:36.311052 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:31:36.311014    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/045d6242-b766-45ce-ae02-6fe5697e71bf-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "045d6242-b766-45ce-ae02-6fe5697e71bf" (UID: "045d6242-b766-45ce-ae02-6fe5697e71bf"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 16 11:31:36.311285 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:31:36.311261    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/045d6242-b766-45ce-ae02-6fe5697e71bf-kube-api-access-rsh8q" (OuterVolumeSpecName: "kube-api-access-rsh8q") pod "045d6242-b766-45ce-ae02-6fe5697e71bf" (UID: "045d6242-b766-45ce-ae02-6fe5697e71bf"). InnerVolumeSpecName "kube-api-access-rsh8q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 11:31:36.311330 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:31:36.311263    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/045d6242-b766-45ce-ae02-6fe5697e71bf-istio-token" (OuterVolumeSpecName: "istio-token") pod "045d6242-b766-45ce-ae02-6fe5697e71bf" (UID: "045d6242-b766-45ce-ae02-6fe5697e71bf"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 11:31:36.402156 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:31:36.402124    2583 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/045d6242-b766-45ce-ae02-6fe5697e71bf-credential-socket\") on node \"ip-10-0-129-28.ec2.internal\" DevicePath \"\""
Jul 16 11:31:36.402156 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:31:36.402152    2583 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/045d6242-b766-45ce-ae02-6fe5697e71bf-istio-envoy\") on node \"ip-10-0-129-28.ec2.internal\" DevicePath \"\""
Jul 16 11:31:36.402156 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:31:36.402162    2583 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/045d6242-b766-45ce-ae02-6fe5697e71bf-istio-data\") on node \"ip-10-0-129-28.ec2.internal\" DevicePath \"\""
Jul 16 11:31:36.402394 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:31:36.402170    2583 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/045d6242-b766-45ce-ae02-6fe5697e71bf-istio-podinfo\") on node \"ip-10-0-129-28.ec2.internal\" DevicePath \"\""
Jul 16 11:31:36.402394 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:31:36.402178    2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/045d6242-b766-45ce-ae02-6fe5697e71bf-istio-token\") on node \"ip-10-0-129-28.ec2.internal\" DevicePath \"\""
Jul 16 11:31:36.402394 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:31:36.402185    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rsh8q\" (UniqueName: \"kubernetes.io/projected/045d6242-b766-45ce-ae02-6fe5697e71bf-kube-api-access-rsh8q\") on node \"ip-10-0-129-28.ec2.internal\" DevicePath \"\""
Jul 16 11:31:36.404438 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:31:36.404411    2583 generic.go:356] "Generic (PLEG): container finished" podID="045d6242-b766-45ce-ae02-6fe5697e71bf" containerID="6e3034359f05bb1886e9fcab4686210277b2d6fa97ad66cbb0b31630c60356f9" exitCode=0
Jul 16 11:31:36.404550 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:31:36.404476    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-6c895ad5-openshift-default-585495bfb7-bsfwk"
Jul 16 11:31:36.404550 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:31:36.404503    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-6c895ad5-openshift-default-585495bfb7-bsfwk" event={"ID":"045d6242-b766-45ce-ae02-6fe5697e71bf","Type":"ContainerDied","Data":"6e3034359f05bb1886e9fcab4686210277b2d6fa97ad66cbb0b31630c60356f9"}
Jul 16 11:31:36.404639 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:31:36.404623    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-6c895ad5-openshift-default-585495bfb7-bsfwk" event={"ID":"045d6242-b766-45ce-ae02-6fe5697e71bf","Type":"ContainerDied","Data":"03e62c44cc579c2c7baae43fda547f4e2aafbb3789ed94ad0b8eb5389db97339"}
Jul 16 11:31:36.404678 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:31:36.404629    2583 scope.go:117] "RemoveContainer" containerID="6e3034359f05bb1886e9fcab4686210277b2d6fa97ad66cbb0b31630c60356f9"
Jul 16 11:31:36.414402 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:31:36.414384    2583 scope.go:117] "RemoveContainer" containerID="a7df3902deb1884e211bfcb997eecb3782a4f6b08175ee4abd469b8ac15b0400"
Jul 16 11:31:36.422851 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:31:36.422826    2583 scope.go:117] "RemoveContainer" containerID="6e3034359f05bb1886e9fcab4686210277b2d6fa97ad66cbb0b31630c60356f9"
Jul 16 11:31:36.423165 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:31:36.423141    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6e3034359f05bb1886e9fcab4686210277b2d6fa97ad66cbb0b31630c60356f9\": container with ID starting with 6e3034359f05bb1886e9fcab4686210277b2d6fa97ad66cbb0b31630c60356f9 not found: ID does not exist" containerID="6e3034359f05bb1886e9fcab4686210277b2d6fa97ad66cbb0b31630c60356f9"
Jul 16 11:31:36.423241 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:31:36.423175    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6e3034359f05bb1886e9fcab4686210277b2d6fa97ad66cbb0b31630c60356f9"} err="failed to get container status \"6e3034359f05bb1886e9fcab4686210277b2d6fa97ad66cbb0b31630c60356f9\": rpc error: code = NotFound desc = could not find container \"6e3034359f05bb1886e9fcab4686210277b2d6fa97ad66cbb0b31630c60356f9\": container with ID starting with 6e3034359f05bb1886e9fcab4686210277b2d6fa97ad66cbb0b31630c60356f9 not found: ID does not exist"
Jul 16 11:31:36.423241 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:31:36.423195    2583 scope.go:117] "RemoveContainer" containerID="a7df3902deb1884e211bfcb997eecb3782a4f6b08175ee4abd469b8ac15b0400"
Jul 16 11:31:36.423438 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:31:36.423422    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a7df3902deb1884e211bfcb997eecb3782a4f6b08175ee4abd469b8ac15b0400\": container with ID starting with a7df3902deb1884e211bfcb997eecb3782a4f6b08175ee4abd469b8ac15b0400 not found: ID does not exist" containerID="a7df3902deb1884e211bfcb997eecb3782a4f6b08175ee4abd469b8ac15b0400"
Jul 16 11:31:36.423482 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:31:36.423443    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a7df3902deb1884e211bfcb997eecb3782a4f6b08175ee4abd469b8ac15b0400"} err="failed to get container status \"a7df3902deb1884e211bfcb997eecb3782a4f6b08175ee4abd469b8ac15b0400\": rpc error: code = NotFound desc = could not find container \"a7df3902deb1884e211bfcb997eecb3782a4f6b08175ee4abd469b8ac15b0400\": container with ID starting with a7df3902deb1884e211bfcb997eecb3782a4f6b08175ee4abd469b8ac15b0400 not found: ID does not exist"
Jul 16 11:31:36.428247 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:31:36.428203    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-6c895ad5-openshift-default-585495bfb7-bsfwk"]
Jul 16 11:31:36.433284 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:31:36.433258    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-6c895ad5-openshift-default-585495bfb7-bsfwk"]
Jul 16 11:31:37.228749 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:31:37.228716    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="045d6242-b766-45ce-ae02-6fe5697e71bf" path="/var/lib/kubelet/pods/045d6242-b766-45ce-ae02-6fe5697e71bf/volumes"
Jul 16 11:32:46.962507 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:32:46.962468    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-bb5ebc-openshift-default-7c4d969fc5-958jr"]
Jul 16 11:32:46.963020 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:32:46.962990    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5fa9839e-d970-47d8-88a6-1c041f8a7eb5" containerName="maas-api"
Jul 16 11:32:46.963020 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:32:46.963008    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="5fa9839e-d970-47d8-88a6-1c041f8a7eb5" containerName="maas-api"
Jul 16 11:32:46.963020 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:32:46.963021    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="045d6242-b766-45ce-ae02-6fe5697e71bf" containerName="istio-proxy"
Jul 16 11:32:46.963126 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:32:46.963027    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="045d6242-b766-45ce-ae02-6fe5697e71bf" containerName="istio-proxy"
Jul 16 11:32:46.963126 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:32:46.963038    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="045d6242-b766-45ce-ae02-6fe5697e71bf" containerName="istio-proxy"
Jul 16 11:32:46.963126 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:32:46.963044    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="045d6242-b766-45ce-ae02-6fe5697e71bf" containerName="istio-proxy"
Jul 16 11:32:46.963126 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:32:46.963107    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="5fa9839e-d970-47d8-88a6-1c041f8a7eb5" containerName="maas-api"
Jul 16 11:32:46.963126 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:32:46.963114    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="045d6242-b766-45ce-ae02-6fe5697e71bf" containerName="istio-proxy"
Jul 16 11:32:46.963126 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:32:46.963121    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="045d6242-b766-45ce-ae02-6fe5697e71bf" containerName="istio-proxy"
Jul 16 11:32:46.967425 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:32:46.967404    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-bb5ebc-openshift-default-7c4d969fc5-958jr"
Jul 16 11:32:46.970403 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:32:46.970385    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-bb5ebc-openshift-default-dockercfg-vggrk\""
Jul 16 11:32:46.984358 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:32:46.984328    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-bb5ebc-openshift-default-7c4d969fc5-958jr"]
Jul 16 11:32:47.028842 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:32:47.028800    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d86cfcb1-6d56-4036-968f-f517e9d0d578-workload-certs\") pod \"e2e-inf-a-bb5ebc-openshift-default-7c4d969fc5-958jr\" (UID: \"d86cfcb1-6d56-4036-968f-f517e9d0d578\") " pod="openshift-ingress/e2e-inf-a-bb5ebc-openshift-default-7c4d969fc5-958jr"
Jul 16 11:32:47.028842 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:32:47.028843    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d86cfcb1-6d56-4036-968f-f517e9d0d578-istiod-ca-cert\") pod \"e2e-inf-a-bb5ebc-openshift-default-7c4d969fc5-958jr\" (UID: \"d86cfcb1-6d56-4036-968f-f517e9d0d578\") " pod="openshift-ingress/e2e-inf-a-bb5ebc-openshift-default-7c4d969fc5-958jr"
Jul 16 11:32:47.029059 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:32:47.028863    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d86cfcb1-6d56-4036-968f-f517e9d0d578-istio-data\") pod \"e2e-inf-a-bb5ebc-openshift-default-7c4d969fc5-958jr\" (UID: \"d86cfcb1-6d56-4036-968f-f517e9d0d578\") " pod="openshift-ingress/e2e-inf-a-bb5ebc-openshift-default-7c4d969fc5-958jr"
Jul 16 11:32:47.029059 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:32:47.028979    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d86cfcb1-6d56-4036-968f-f517e9d0d578-istio-envoy\") pod \"e2e-inf-a-bb5ebc-openshift-default-7c4d969fc5-958jr\" (UID: \"d86cfcb1-6d56-4036-968f-f517e9d0d578\") " pod="openshift-ingress/e2e-inf-a-bb5ebc-openshift-default-7c4d969fc5-958jr"
Jul 16 11:32:47.029059 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:32:47.029011    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d86cfcb1-6d56-4036-968f-f517e9d0d578-workload-socket\") pod \"e2e-inf-a-bb5ebc-openshift-default-7c4d969fc5-958jr\" (UID: \"d86cfcb1-6d56-4036-968f-f517e9d0d578\") " pod="openshift-ingress/e2e-inf-a-bb5ebc-openshift-default-7c4d969fc5-958jr"
Jul 16 11:32:47.029059 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:32:47.029049    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d86cfcb1-6d56-4036-968f-f517e9d0d578-credential-socket\") pod \"e2e-inf-a-bb5ebc-openshift-default-7c4d969fc5-958jr\" (UID: \"d86cfcb1-6d56-4036-968f-f517e9d0d578\") " pod="openshift-ingress/e2e-inf-a-bb5ebc-openshift-default-7c4d969fc5-958jr"
Jul 16 11:32:47.029194 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:32:47.029079    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ss8zs\" (UniqueName: \"kubernetes.io/projected/d86cfcb1-6d56-4036-968f-f517e9d0d578-kube-api-access-ss8zs\") pod \"e2e-inf-a-bb5ebc-openshift-default-7c4d969fc5-958jr\" (UID: \"d86cfcb1-6d56-4036-968f-f517e9d0d578\") " pod="openshift-ingress/e2e-inf-a-bb5ebc-openshift-default-7c4d969fc5-958jr"
Jul 16 11:32:47.029194 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:32:47.029105    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d86cfcb1-6d56-4036-968f-f517e9d0d578-istio-token\") pod \"e2e-inf-a-bb5ebc-openshift-default-7c4d969fc5-958jr\" (UID: \"d86cfcb1-6d56-4036-968f-f517e9d0d578\") " pod="openshift-ingress/e2e-inf-a-bb5ebc-openshift-default-7c4d969fc5-958jr"
Jul 16 11:32:47.029194 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:32:47.029132    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d86cfcb1-6d56-4036-968f-f517e9d0d578-istio-podinfo\") pod \"e2e-inf-a-bb5ebc-openshift-default-7c4d969fc5-958jr\" (UID: \"d86cfcb1-6d56-4036-968f-f517e9d0d578\") " pod="openshift-ingress/e2e-inf-a-bb5ebc-openshift-default-7c4d969fc5-958jr"
Jul 16 11:32:47.130401 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:32:47.130356    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d86cfcb1-6d56-4036-968f-f517e9d0d578-credential-socket\") pod \"e2e-inf-a-bb5ebc-openshift-default-7c4d969fc5-958jr\" (UID: \"d86cfcb1-6d56-4036-968f-f517e9d0d578\") " pod="openshift-ingress/e2e-inf-a-bb5ebc-openshift-default-7c4d969fc5-958jr"
Jul 16 11:32:47.130401 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:32:47.130405    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ss8zs\" (UniqueName: \"kubernetes.io/projected/d86cfcb1-6d56-4036-968f-f517e9d0d578-kube-api-access-ss8zs\") pod \"e2e-inf-a-bb5ebc-openshift-default-7c4d969fc5-958jr\" (UID: \"d86cfcb1-6d56-4036-968f-f517e9d0d578\") " pod="openshift-ingress/e2e-inf-a-bb5ebc-openshift-default-7c4d969fc5-958jr"
Jul 16 11:32:47.130683 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:32:47.130443    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d86cfcb1-6d56-4036-968f-f517e9d0d578-istio-token\") pod \"e2e-inf-a-bb5ebc-openshift-default-7c4d969fc5-958jr\" (UID: \"d86cfcb1-6d56-4036-968f-f517e9d0d578\") " pod="openshift-ingress/e2e-inf-a-bb5ebc-openshift-default-7c4d969fc5-958jr"
Jul 16 11:32:47.130683 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:32:47.130470    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d86cfcb1-6d56-4036-968f-f517e9d0d578-istio-podinfo\") pod \"e2e-inf-a-bb5ebc-openshift-default-7c4d969fc5-958jr\" (UID: \"d86cfcb1-6d56-4036-968f-f517e9d0d578\") " pod="openshift-ingress/e2e-inf-a-bb5ebc-openshift-default-7c4d969fc5-958jr"
Jul 16 11:32:47.130683 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:32:47.130521    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d86cfcb1-6d56-4036-968f-f517e9d0d578-workload-certs\") pod \"e2e-inf-a-bb5ebc-openshift-default-7c4d969fc5-958jr\" (UID: \"d86cfcb1-6d56-4036-968f-f517e9d0d578\") " pod="openshift-ingress/e2e-inf-a-bb5ebc-openshift-default-7c4d969fc5-958jr"
Jul 16 11:32:47.130683 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:32:47.130549    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d86cfcb1-6d56-4036-968f-f517e9d0d578-istiod-ca-cert\") pod \"e2e-inf-a-bb5ebc-openshift-default-7c4d969fc5-958jr\" (UID: \"d86cfcb1-6d56-4036-968f-f517e9d0d578\") " pod="openshift-ingress/e2e-inf-a-bb5ebc-openshift-default-7c4d969fc5-958jr"
Jul 16 11:32:47.130683 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:32:47.130576    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d86cfcb1-6d56-4036-968f-f517e9d0d578-istio-data\") pod \"e2e-inf-a-bb5ebc-openshift-default-7c4d969fc5-958jr\" (UID: \"d86cfcb1-6d56-4036-968f-f517e9d0d578\") " pod="openshift-ingress/e2e-inf-a-bb5ebc-openshift-default-7c4d969fc5-958jr"
Jul 16 11:32:47.130683 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:32:47.130673    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d86cfcb1-6d56-4036-968f-f517e9d0d578-istio-envoy\") pod \"e2e-inf-a-bb5ebc-openshift-default-7c4d969fc5-958jr\" (UID: \"d86cfcb1-6d56-4036-968f-f517e9d0d578\") " pod="openshift-ingress/e2e-inf-a-bb5ebc-openshift-default-7c4d969fc5-958jr"
Jul 16 11:32:47.130929 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:32:47.130702    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d86cfcb1-6d56-4036-968f-f517e9d0d578-workload-socket\") pod \"e2e-inf-a-bb5ebc-openshift-default-7c4d969fc5-958jr\" (UID: \"d86cfcb1-6d56-4036-968f-f517e9d0d578\") " pod="openshift-ingress/e2e-inf-a-bb5ebc-openshift-default-7c4d969fc5-958jr"
Jul 16 11:32:47.130982 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:32:47.130921    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d86cfcb1-6d56-4036-968f-f517e9d0d578-credential-socket\") pod \"e2e-inf-a-bb5ebc-openshift-default-7c4d969fc5-958jr\" (UID: \"d86cfcb1-6d56-4036-968f-f517e9d0d578\") " pod="openshift-ingress/e2e-inf-a-bb5ebc-openshift-default-7c4d969fc5-958jr"
Jul 16 11:32:47.131035 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:32:47.130997    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d86cfcb1-6d56-4036-968f-f517e9d0d578-workload-certs\") pod \"e2e-inf-a-bb5ebc-openshift-default-7c4d969fc5-958jr\" (UID: \"d86cfcb1-6d56-4036-968f-f517e9d0d578\") " pod="openshift-ingress/e2e-inf-a-bb5ebc-openshift-default-7c4d969fc5-958jr"
Jul 16 11:32:47.131084 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:32:47.131044    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d86cfcb1-6d56-4036-968f-f517e9d0d578-istio-data\") pod \"e2e-inf-a-bb5ebc-openshift-default-7c4d969fc5-958jr\" (UID: \"d86cfcb1-6d56-4036-968f-f517e9d0d578\") " pod="openshift-ingress/e2e-inf-a-bb5ebc-openshift-default-7c4d969fc5-958jr"
Jul 16 11:32:47.131164 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:32:47.131145    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d86cfcb1-6d56-4036-968f-f517e9d0d578-workload-socket\") pod \"e2e-inf-a-bb5ebc-openshift-default-7c4d969fc5-958jr\" (UID: \"d86cfcb1-6d56-4036-968f-f517e9d0d578\") " pod="openshift-ingress/e2e-inf-a-bb5ebc-openshift-default-7c4d969fc5-958jr"
Jul 16 11:32:47.131553 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:32:47.131529    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d86cfcb1-6d56-4036-968f-f517e9d0d578-istiod-ca-cert\") pod \"e2e-inf-a-bb5ebc-openshift-default-7c4d969fc5-958jr\" (UID: \"d86cfcb1-6d56-4036-968f-f517e9d0d578\") " pod="openshift-ingress/e2e-inf-a-bb5ebc-openshift-default-7c4d969fc5-958jr"
Jul 16 11:32:47.132997 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:32:47.132973    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d86cfcb1-6d56-4036-968f-f517e9d0d578-istio-envoy\") pod \"e2e-inf-a-bb5ebc-openshift-default-7c4d969fc5-958jr\" (UID: \"d86cfcb1-6d56-4036-968f-f517e9d0d578\") " pod="openshift-ingress/e2e-inf-a-bb5ebc-openshift-default-7c4d969fc5-958jr"
Jul 16 11:32:47.133239 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:32:47.133223    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d86cfcb1-6d56-4036-968f-f517e9d0d578-istio-podinfo\") pod \"e2e-inf-a-bb5ebc-openshift-default-7c4d969fc5-958jr\" (UID: \"d86cfcb1-6d56-4036-968f-f517e9d0d578\") " pod="openshift-ingress/e2e-inf-a-bb5ebc-openshift-default-7c4d969fc5-958jr"
Jul 16 11:32:47.142008 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:32:47.141976    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d86cfcb1-6d56-4036-968f-f517e9d0d578-istio-token\") pod \"e2e-inf-a-bb5ebc-openshift-default-7c4d969fc5-958jr\" (UID: \"d86cfcb1-6d56-4036-968f-f517e9d0d578\") " pod="openshift-ingress/e2e-inf-a-bb5ebc-openshift-default-7c4d969fc5-958jr"
Jul 16 11:32:47.142404 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:32:47.142386    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ss8zs\" (UniqueName: \"kubernetes.io/projected/d86cfcb1-6d56-4036-968f-f517e9d0d578-kube-api-access-ss8zs\") pod \"e2e-inf-a-bb5ebc-openshift-default-7c4d969fc5-958jr\" (UID: \"d86cfcb1-6d56-4036-968f-f517e9d0d578\") " pod="openshift-ingress/e2e-inf-a-bb5ebc-openshift-default-7c4d969fc5-958jr"
Jul 16 11:32:47.277484 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:32:47.277390    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-bb5ebc-openshift-default-7c4d969fc5-958jr"
Jul 16 11:32:47.419391 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:32:47.419367    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-bb5ebc-openshift-default-7c4d969fc5-958jr"]
Jul 16 11:32:47.421414 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:32:47.421389    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd86cfcb1_6d56_4036_968f_f517e9d0d578.slice/crio-d468626e7d0ea9c998f85f99d63da3065773eb2c013123c5478dc1678b01237a WatchSource:0}: Error finding container d468626e7d0ea9c998f85f99d63da3065773eb2c013123c5478dc1678b01237a: Status 404 returned error can't find the container with id d468626e7d0ea9c998f85f99d63da3065773eb2c013123c5478dc1678b01237a
Jul 16 11:32:47.423503 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:32:47.423471    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 11:32:47.423623 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:32:47.423538    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 11:32:47.423623 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:32:47.423570    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 11:32:47.749105 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:32:47.749062    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-bb5ebc-openshift-default-7c4d969fc5-958jr" event={"ID":"d86cfcb1-6d56-4036-968f-f517e9d0d578","Type":"ContainerStarted","Data":"597a9426b78dcb0c21641de8b2e8ac132c46e852d4e32df0c9ae619cb015d8a5"}
Jul 16 11:32:47.749306 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:32:47.749122    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-bb5ebc-openshift-default-7c4d969fc5-958jr" event={"ID":"d86cfcb1-6d56-4036-968f-f517e9d0d578","Type":"ContainerStarted","Data":"d468626e7d0ea9c998f85f99d63da3065773eb2c013123c5478dc1678b01237a"}
Jul 16 11:32:47.772870 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:32:47.772816    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-bb5ebc-openshift-default-7c4d969fc5-958jr" podStartSLOduration=1.7727995349999999 podStartE2EDuration="1.772799535s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 11:32:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 11:32:47.771791154 +0000 UTC m=+1760.406012632" watchObservedRunningTime="2026-07-16 11:32:47.772799535 +0000 UTC m=+1760.407020989"
Jul 16 11:32:48.277562 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:32:48.277529    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-bb5ebc-openshift-default-7c4d969fc5-958jr"
Jul 16 11:32:48.282452 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:32:48.282427    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-bb5ebc-openshift-default-7c4d969fc5-958jr"
Jul 16 11:32:48.755584 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:32:48.755551    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-bb5ebc-openshift-default-7c4d969fc5-958jr"
Jul 16 11:32:48.756949 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:32:48.756928    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-bb5ebc-openshift-default-7c4d969fc5-958jr"
Jul 16 11:32:49.018776 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:32:49.018696    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-bb5ebc-568cd484b-b46v5"]
Jul 16 11:32:49.022305 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:32:49.022287    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-bb5ebc-568cd484b-b46v5"
Jul 16 11:32:49.025968 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:32:49.025942    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-bb5ebc-serving-cert\""
Jul 16 11:32:49.039811 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:32:49.039780    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-bb5ebc-568cd484b-b46v5"]
Jul 16 11:32:49.048062 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:32:49.048035    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j4xnn\" (UniqueName: \"kubernetes.io/projected/2890a09f-e805-4f7d-8752-6cdc7b826206-kube-api-access-j4xnn\") pod \"maas-api-e2e-inf-a-bb5ebc-568cd484b-b46v5\" (UID: \"2890a09f-e805-4f7d-8752-6cdc7b826206\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-bb5ebc-568cd484b-b46v5"
Jul 16 11:32:49.048197 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:32:49.048071    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2890a09f-e805-4f7d-8752-6cdc7b826206-openshift-service-ca\") pod \"maas-api-e2e-inf-a-bb5ebc-568cd484b-b46v5\" (UID: \"2890a09f-e805-4f7d-8752-6cdc7b826206\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-bb5ebc-568cd484b-b46v5"
Jul 16 11:32:49.048197 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:32:49.048162    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2890a09f-e805-4f7d-8752-6cdc7b826206-maas-api-tls\") pod \"maas-api-e2e-inf-a-bb5ebc-568cd484b-b46v5\" (UID: \"2890a09f-e805-4f7d-8752-6cdc7b826206\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-bb5ebc-568cd484b-b46v5"
Jul 16 11:32:49.148799 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:32:49.148760    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j4xnn\" (UniqueName: \"kubernetes.io/projected/2890a09f-e805-4f7d-8752-6cdc7b826206-kube-api-access-j4xnn\") pod \"maas-api-e2e-inf-a-bb5ebc-568cd484b-b46v5\" (UID: \"2890a09f-e805-4f7d-8752-6cdc7b826206\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-bb5ebc-568cd484b-b46v5"
Jul 16 11:32:49.148799 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:32:49.148805    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2890a09f-e805-4f7d-8752-6cdc7b826206-openshift-service-ca\") pod \"maas-api-e2e-inf-a-bb5ebc-568cd484b-b46v5\" (UID: \"2890a09f-e805-4f7d-8752-6cdc7b826206\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-bb5ebc-568cd484b-b46v5"
Jul 16 11:32:49.149045 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:32:49.148842    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2890a09f-e805-4f7d-8752-6cdc7b826206-maas-api-tls\") pod \"maas-api-e2e-inf-a-bb5ebc-568cd484b-b46v5\" (UID: \"2890a09f-e805-4f7d-8752-6cdc7b826206\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-bb5ebc-568cd484b-b46v5"
Jul 16 11:32:49.149540 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:32:49.149509    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2890a09f-e805-4f7d-8752-6cdc7b826206-openshift-service-ca\") pod \"maas-api-e2e-inf-a-bb5ebc-568cd484b-b46v5\" (UID: \"2890a09f-e805-4f7d-8752-6cdc7b826206\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-bb5ebc-568cd484b-b46v5"
Jul 16 11:32:49.151261 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:32:49.151235    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2890a09f-e805-4f7d-8752-6cdc7b826206-maas-api-tls\") pod \"maas-api-e2e-inf-a-bb5ebc-568cd484b-b46v5\" (UID: \"2890a09f-e805-4f7d-8752-6cdc7b826206\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-bb5ebc-568cd484b-b46v5"
Jul 16 11:32:49.157734 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:32:49.157708    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j4xnn\" (UniqueName: \"kubernetes.io/projected/2890a09f-e805-4f7d-8752-6cdc7b826206-kube-api-access-j4xnn\") pod \"maas-api-e2e-inf-a-bb5ebc-568cd484b-b46v5\" (UID: \"2890a09f-e805-4f7d-8752-6cdc7b826206\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-bb5ebc-568cd484b-b46v5"
Jul 16 11:32:49.333206 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:32:49.333099    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-bb5ebc-568cd484b-b46v5"
Jul 16 11:32:49.473856 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:32:49.473826    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-bb5ebc-568cd484b-b46v5"]
Jul 16 11:32:49.476042 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:32:49.476007    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2890a09f_e805_4f7d_8752_6cdc7b826206.slice/crio-117fa6205ba23d951c5372497068d650cd8155e9e9105736fe4e4e3ef20e3017 WatchSource:0}: Error finding container 117fa6205ba23d951c5372497068d650cd8155e9e9105736fe4e4e3ef20e3017: Status 404 returned error can't find the container with id 117fa6205ba23d951c5372497068d650cd8155e9e9105736fe4e4e3ef20e3017
Jul 16 11:32:49.761697 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:32:49.761657    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-bb5ebc-568cd484b-b46v5" event={"ID":"2890a09f-e805-4f7d-8752-6cdc7b826206","Type":"ContainerStarted","Data":"117fa6205ba23d951c5372497068d650cd8155e9e9105736fe4e4e3ef20e3017"}
Jul 16 11:32:50.767995 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:32:50.767960    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-bb5ebc-568cd484b-b46v5" event={"ID":"2890a09f-e805-4f7d-8752-6cdc7b826206","Type":"ContainerStarted","Data":"6abf825e0c853aa5ef0c57b295f03d0c4f102296666a014446a9aaa99aee3404"}
Jul 16 11:32:50.768430 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:32:50.768120    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-bb5ebc-568cd484b-b46v5"
Jul 16 11:32:50.791876 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:32:50.791818    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-bb5ebc-568cd484b-b46v5" podStartSLOduration=2.240788099 podStartE2EDuration="2.791800749s" totalImagesPullingTime="551.01265ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 11:32:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 11:32:50.791688056 +0000 UTC m=+1763.425909512" watchObservedRunningTime="2026-07-16 11:32:50.791800749 +0000 UTC m=+1763.426022203"
Jul 16 11:32:56.779081 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:32:56.779054    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-bb5ebc-568cd484b-b46v5"
Jul 16 11:32:57.379865 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:32:57.379829    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-96e676-bdcd69b49-7jmxp"]
Jul 16 11:32:57.385073 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:32:57.385043    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-96e676-bdcd69b49-7jmxp"
Jul 16 11:32:57.387716 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:32:57.387686    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-96e676-serving-cert\""
Jul 16 11:32:57.410250 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:32:57.410218    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-96e676-bdcd69b49-7jmxp"]
Jul 16 11:32:57.426750 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:32:57.426715    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dz682\" (UniqueName: \"kubernetes.io/projected/e119146e-22e0-4d10-8bb9-5dd566b033e8-kube-api-access-dz682\") pod \"maas-api-e2e-inf-b-96e676-bdcd69b49-7jmxp\" (UID: \"e119146e-22e0-4d10-8bb9-5dd566b033e8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-96e676-bdcd69b49-7jmxp"
Jul 16 11:32:57.427044 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:32:57.426768    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e119146e-22e0-4d10-8bb9-5dd566b033e8-maas-api-tls\") pod \"maas-api-e2e-inf-b-96e676-bdcd69b49-7jmxp\" (UID: \"e119146e-22e0-4d10-8bb9-5dd566b033e8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-96e676-bdcd69b49-7jmxp"
Jul 16 11:32:57.427044 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:32:57.426832    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e119146e-22e0-4d10-8bb9-5dd566b033e8-openshift-service-ca\") pod \"maas-api-e2e-inf-b-96e676-bdcd69b49-7jmxp\" (UID: \"e119146e-22e0-4d10-8bb9-5dd566b033e8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-96e676-bdcd69b49-7jmxp"
Jul 16 11:32:57.528302 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:32:57.528263    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e119146e-22e0-4d10-8bb9-5dd566b033e8-maas-api-tls\") pod \"maas-api-e2e-inf-b-96e676-bdcd69b49-7jmxp\" (UID: \"e119146e-22e0-4d10-8bb9-5dd566b033e8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-96e676-bdcd69b49-7jmxp"
Jul 16 11:32:57.528524 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:32:57.528328    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e119146e-22e0-4d10-8bb9-5dd566b033e8-openshift-service-ca\") pod \"maas-api-e2e-inf-b-96e676-bdcd69b49-7jmxp\" (UID: \"e119146e-22e0-4d10-8bb9-5dd566b033e8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-96e676-bdcd69b49-7jmxp"
Jul 16 11:32:57.528524 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:32:57.528404    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dz682\" (UniqueName: \"kubernetes.io/projected/e119146e-22e0-4d10-8bb9-5dd566b033e8-kube-api-access-dz682\") pod \"maas-api-e2e-inf-b-96e676-bdcd69b49-7jmxp\" (UID: \"e119146e-22e0-4d10-8bb9-5dd566b033e8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-96e676-bdcd69b49-7jmxp"
Jul 16 11:32:57.528524 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:32:57.528431    2583 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-96e676-serving-cert: secret "maas-api-e2e-inf-b-96e676-serving-cert" not found
Jul 16 11:32:57.528524 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:32:57.528498    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e119146e-22e0-4d10-8bb9-5dd566b033e8-maas-api-tls podName:e119146e-22e0-4d10-8bb9-5dd566b033e8 nodeName:}" failed. No retries permitted until 2026-07-16 11:32:58.028479995 +0000 UTC m=+1770.662701430 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/e119146e-22e0-4d10-8bb9-5dd566b033e8-maas-api-tls") pod "maas-api-e2e-inf-b-96e676-bdcd69b49-7jmxp" (UID: "e119146e-22e0-4d10-8bb9-5dd566b033e8") : secret "maas-api-e2e-inf-b-96e676-serving-cert" not found
Jul 16 11:32:57.529157 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:32:57.529136    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e119146e-22e0-4d10-8bb9-5dd566b033e8-openshift-service-ca\") pod \"maas-api-e2e-inf-b-96e676-bdcd69b49-7jmxp\" (UID: \"e119146e-22e0-4d10-8bb9-5dd566b033e8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-96e676-bdcd69b49-7jmxp"
Jul 16 11:32:57.542123 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:32:57.542094    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dz682\" (UniqueName: \"kubernetes.io/projected/e119146e-22e0-4d10-8bb9-5dd566b033e8-kube-api-access-dz682\") pod \"maas-api-e2e-inf-b-96e676-bdcd69b49-7jmxp\" (UID: \"e119146e-22e0-4d10-8bb9-5dd566b033e8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-96e676-bdcd69b49-7jmxp"
Jul 16 11:32:58.034025 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:32:58.033986    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e119146e-22e0-4d10-8bb9-5dd566b033e8-maas-api-tls\") pod \"maas-api-e2e-inf-b-96e676-bdcd69b49-7jmxp\" (UID: \"e119146e-22e0-4d10-8bb9-5dd566b033e8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-96e676-bdcd69b49-7jmxp"
Jul 16 11:32:58.036467 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:32:58.036434    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e119146e-22e0-4d10-8bb9-5dd566b033e8-maas-api-tls\") pod \"maas-api-e2e-inf-b-96e676-bdcd69b49-7jmxp\" (UID: \"e119146e-22e0-4d10-8bb9-5dd566b033e8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-96e676-bdcd69b49-7jmxp"
Jul 16 11:32:58.310429 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:32:58.310338    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-96e676-bdcd69b49-7jmxp"
Jul 16 11:32:58.456979 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:32:58.456948    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-96e676-bdcd69b49-7jmxp"]
Jul 16 11:32:58.459449 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:32:58.459415    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode119146e_22e0_4d10_8bb9_5dd566b033e8.slice/crio-f45e09e372738604ea3a0ea5ff022faf0e0e2dfaa54f07dfaa69e585eb1e32db WatchSource:0}: Error finding container f45e09e372738604ea3a0ea5ff022faf0e0e2dfaa54f07dfaa69e585eb1e32db: Status 404 returned error can't find the container with id f45e09e372738604ea3a0ea5ff022faf0e0e2dfaa54f07dfaa69e585eb1e32db
Jul 16 11:32:58.805614 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:32:58.805571    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-96e676-bdcd69b49-7jmxp" event={"ID":"e119146e-22e0-4d10-8bb9-5dd566b033e8","Type":"ContainerStarted","Data":"f45e09e372738604ea3a0ea5ff022faf0e0e2dfaa54f07dfaa69e585eb1e32db"}
Jul 16 11:32:59.812218 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:32:59.812182    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-96e676-bdcd69b49-7jmxp" event={"ID":"e119146e-22e0-4d10-8bb9-5dd566b033e8","Type":"ContainerStarted","Data":"f1a13a5b396a64b1b71bcc24057cf4a625bfc680f9377cb9f65c006363c0d908"}
Jul 16 11:32:59.812702 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:32:59.812313    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-96e676-bdcd69b49-7jmxp"
Jul 16 11:32:59.831990 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:32:59.831921    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-96e676-bdcd69b49-7jmxp" podStartSLOduration=2.42506876 podStartE2EDuration="2.831898203s" totalImagesPullingTime="406.829443ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 11:32:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 11:32:59.829231263 +0000 UTC m=+1772.463452736" watchObservedRunningTime="2026-07-16 11:32:59.831898203 +0000 UTC m=+1772.466119658"
Jul 16 11:33:04.797793 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:33:04.797742    2583 patch_prober.go:28] interesting pod/e2e-inf-a-bb5ebc-openshift-default-7c4d969fc5-958jr container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 16 11:33:04.798292 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:33:04.797811    2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-bb5ebc-openshift-default-7c4d969fc5-958jr" podUID="d86cfcb1-6d56-4036-968f-f517e9d0d578" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 16 11:33:05.823678 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:33:05.823649    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-96e676-bdcd69b49-7jmxp"
Jul 16 11:33:06.853120 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:33:06.853085    2583 generic.go:356] "Generic (PLEG): container finished" podID="d86cfcb1-6d56-4036-968f-f517e9d0d578" containerID="597a9426b78dcb0c21641de8b2e8ac132c46e852d4e32df0c9ae619cb015d8a5" exitCode=0
Jul 16 11:33:06.853676 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:33:06.853167    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-bb5ebc-openshift-default-7c4d969fc5-958jr" event={"ID":"d86cfcb1-6d56-4036-968f-f517e9d0d578","Type":"ContainerDied","Data":"597a9426b78dcb0c21641de8b2e8ac132c46e852d4e32df0c9ae619cb015d8a5"}
Jul 16 11:33:06.853749 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:33:06.853724    2583 scope.go:117] "RemoveContainer" containerID="597a9426b78dcb0c21641de8b2e8ac132c46e852d4e32df0c9ae619cb015d8a5"
Jul 16 11:33:06.854752 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:33:06.854726    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 11:33:06.854866 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:33:06.854805    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 11:33:06.854866 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:33:06.854852    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 11:33:07.860275 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:33:07.860239    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-bb5ebc-openshift-default-7c4d969fc5-958jr" event={"ID":"d86cfcb1-6d56-4036-968f-f517e9d0d578","Type":"ContainerStarted","Data":"54c404e072934a9f6f68607e7696f47812407344704ee649b30ea4d64ab56779"}
Jul 16 11:33:08.277590 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:33:08.277552    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-bb5ebc-openshift-default-7c4d969fc5-958jr"
Jul 16 11:33:08.277809 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:33:08.277783    2583 patch_prober.go:28] interesting pod/e2e-inf-a-bb5ebc-openshift-default-7c4d969fc5-958jr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body=
Jul 16 11:33:08.277856 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:33:08.277828    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-bb5ebc-openshift-default-7c4d969fc5-958jr" podUID="d86cfcb1-6d56-4036-968f-f517e9d0d578" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused"
Jul 16 11:33:09.284192 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:33:09.278251    2583 patch_prober.go:28] interesting pod/e2e-inf-a-bb5ebc-openshift-default-7c4d969fc5-958jr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body=
Jul 16 11:33:09.284192 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:33:09.278772    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-bb5ebc-openshift-default-7c4d969fc5-958jr" podUID="d86cfcb1-6d56-4036-968f-f517e9d0d578" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused"
Jul 16 11:33:10.278762 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:33:10.278719    2583 patch_prober.go:28] interesting pod/e2e-inf-a-bb5ebc-openshift-default-7c4d969fc5-958jr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body=
Jul 16 11:33:10.278960 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:33:10.278781    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-bb5ebc-openshift-default-7c4d969fc5-958jr" podUID="d86cfcb1-6d56-4036-968f-f517e9d0d578" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused"
Jul 16 11:33:11.278708 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:33:11.278671    2583 patch_prober.go:28] interesting pod/e2e-inf-a-bb5ebc-openshift-default-7c4d969fc5-958jr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body=
Jul 16 11:33:11.279181 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:33:11.278735    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-bb5ebc-openshift-default-7c4d969fc5-958jr" podUID="d86cfcb1-6d56-4036-968f-f517e9d0d578" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused"
Jul 16 11:33:12.278471 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:33:12.278429    2583 patch_prober.go:28] interesting pod/e2e-inf-a-bb5ebc-openshift-default-7c4d969fc5-958jr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body=
Jul 16 11:33:12.278710 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:33:12.278508    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-bb5ebc-openshift-default-7c4d969fc5-958jr" podUID="d86cfcb1-6d56-4036-968f-f517e9d0d578" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused"
Jul 16 11:33:13.282462 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:33:13.282431    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-bb5ebc-openshift-default-7c4d969fc5-958jr"
Jul 16 11:33:13.282881 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:33:13.282817    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-bb5ebc-openshift-default-7c4d969fc5-958jr"
Jul 16 11:33:13.283931 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:33:13.283910    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-bb5ebc-openshift-default-7c4d969fc5-958jr"
Jul 16 11:33:29.256016 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:33:29.255992    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-9lm5s_f64f15a3-5a9f-4737-bc4f-73d5ec311e41/console-operator/1.log"
Jul 16 11:33:29.261133 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:33:29.261109    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-9lm5s_f64f15a3-5a9f-4737-bc4f-73d5ec311e41/console-operator/1.log"
Jul 16 11:33:29.262210 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:33:29.262188    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2db6t_69aed169-e8af-429d-99c3-3b20be64e351/ovn-acl-logging/0.log"
Jul 16 11:33:29.267056 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:33:29.267038    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2db6t_69aed169-e8af-429d-99c3-3b20be64e351/ovn-acl-logging/0.log"
Jul 16 11:35:09.887930 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:09.887896    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-bb5ebc-568cd484b-b46v5"]
Jul 16 11:35:09.888457 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:09.888093    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-bb5ebc-568cd484b-b46v5" podUID="2890a09f-e805-4f7d-8752-6cdc7b826206" containerName="maas-api" containerID="cri-o://6abf825e0c853aa5ef0c57b295f03d0c4f102296666a014446a9aaa99aee3404" gracePeriod=30
Jul 16 11:35:10.147437 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:10.147409    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-bb5ebc-568cd484b-b46v5"
Jul 16 11:35:10.255157 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:10.255118    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j4xnn\" (UniqueName: \"kubernetes.io/projected/2890a09f-e805-4f7d-8752-6cdc7b826206-kube-api-access-j4xnn\") pod \"2890a09f-e805-4f7d-8752-6cdc7b826206\" (UID: \"2890a09f-e805-4f7d-8752-6cdc7b826206\") "
Jul 16 11:35:10.255371 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:10.255247    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2890a09f-e805-4f7d-8752-6cdc7b826206-openshift-service-ca\") pod \"2890a09f-e805-4f7d-8752-6cdc7b826206\" (UID: \"2890a09f-e805-4f7d-8752-6cdc7b826206\") "
Jul 16 11:35:10.255371 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:10.255274    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2890a09f-e805-4f7d-8752-6cdc7b826206-maas-api-tls\") pod \"2890a09f-e805-4f7d-8752-6cdc7b826206\" (UID: \"2890a09f-e805-4f7d-8752-6cdc7b826206\") "
Jul 16 11:35:10.255630 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:10.255573    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2890a09f-e805-4f7d-8752-6cdc7b826206-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "2890a09f-e805-4f7d-8752-6cdc7b826206" (UID: "2890a09f-e805-4f7d-8752-6cdc7b826206"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 11:35:10.257352 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:10.257321    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2890a09f-e805-4f7d-8752-6cdc7b826206-kube-api-access-j4xnn" (OuterVolumeSpecName: "kube-api-access-j4xnn") pod "2890a09f-e805-4f7d-8752-6cdc7b826206" (UID: "2890a09f-e805-4f7d-8752-6cdc7b826206"). InnerVolumeSpecName "kube-api-access-j4xnn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 11:35:10.257450 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:10.257348    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2890a09f-e805-4f7d-8752-6cdc7b826206-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "2890a09f-e805-4f7d-8752-6cdc7b826206" (UID: "2890a09f-e805-4f7d-8752-6cdc7b826206"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 11:35:10.355972 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:10.355934    2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2890a09f-e805-4f7d-8752-6cdc7b826206-openshift-service-ca\") on node \"ip-10-0-129-28.ec2.internal\" DevicePath \"\""
Jul 16 11:35:10.355972 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:10.355966    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2890a09f-e805-4f7d-8752-6cdc7b826206-maas-api-tls\") on node \"ip-10-0-129-28.ec2.internal\" DevicePath \"\""
Jul 16 11:35:10.355972 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:10.355975    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j4xnn\" (UniqueName: \"kubernetes.io/projected/2890a09f-e805-4f7d-8752-6cdc7b826206-kube-api-access-j4xnn\") on node \"ip-10-0-129-28.ec2.internal\" DevicePath \"\""
Jul 16 11:35:10.468407 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:10.468318    2583 generic.go:356] "Generic (PLEG): container finished" podID="2890a09f-e805-4f7d-8752-6cdc7b826206" containerID="6abf825e0c853aa5ef0c57b295f03d0c4f102296666a014446a9aaa99aee3404" exitCode=0
Jul 16 11:35:10.468407 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:10.468363    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-bb5ebc-568cd484b-b46v5"
Jul 16 11:35:10.468627 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:10.468404    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-bb5ebc-568cd484b-b46v5" event={"ID":"2890a09f-e805-4f7d-8752-6cdc7b826206","Type":"ContainerDied","Data":"6abf825e0c853aa5ef0c57b295f03d0c4f102296666a014446a9aaa99aee3404"}
Jul 16 11:35:10.468627 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:10.468455    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-bb5ebc-568cd484b-b46v5" event={"ID":"2890a09f-e805-4f7d-8752-6cdc7b826206","Type":"ContainerDied","Data":"117fa6205ba23d951c5372497068d650cd8155e9e9105736fe4e4e3ef20e3017"}
Jul 16 11:35:10.468627 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:10.468483    2583 scope.go:117] "RemoveContainer" containerID="6abf825e0c853aa5ef0c57b295f03d0c4f102296666a014446a9aaa99aee3404"
Jul 16 11:35:10.478236 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:10.478218    2583 scope.go:117] "RemoveContainer" containerID="6abf825e0c853aa5ef0c57b295f03d0c4f102296666a014446a9aaa99aee3404"
Jul 16 11:35:10.478501 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:35:10.478484    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6abf825e0c853aa5ef0c57b295f03d0c4f102296666a014446a9aaa99aee3404\": container with ID starting with 6abf825e0c853aa5ef0c57b295f03d0c4f102296666a014446a9aaa99aee3404 not found: ID does not exist" containerID="6abf825e0c853aa5ef0c57b295f03d0c4f102296666a014446a9aaa99aee3404"
Jul 16 11:35:10.478541 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:10.478512    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6abf825e0c853aa5ef0c57b295f03d0c4f102296666a014446a9aaa99aee3404"} err="failed to get container status \"6abf825e0c853aa5ef0c57b295f03d0c4f102296666a014446a9aaa99aee3404\": rpc error: code = NotFound desc = could not find container \"6abf825e0c853aa5ef0c57b295f03d0c4f102296666a014446a9aaa99aee3404\": container with ID starting with 6abf825e0c853aa5ef0c57b295f03d0c4f102296666a014446a9aaa99aee3404 not found: ID does not exist"
Jul 16 11:35:10.490761 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:10.490732    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-bb5ebc-568cd484b-b46v5"]
Jul 16 11:35:10.493751 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:10.493725    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-bb5ebc-568cd484b-b46v5"]
Jul 16 11:35:11.228681 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:11.228650    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2890a09f-e805-4f7d-8752-6cdc7b826206" path="/var/lib/kubelet/pods/2890a09f-e805-4f7d-8752-6cdc7b826206/volumes"
Jul 16 11:35:17.438176 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:17.438137    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-bb5ebc-openshift-default-7c4d969fc5-958jr"]
Jul 16 11:35:17.438620 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:17.438380    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-bb5ebc-openshift-default-7c4d969fc5-958jr" podUID="d86cfcb1-6d56-4036-968f-f517e9d0d578" containerName="istio-proxy" containerID="cri-o://54c404e072934a9f6f68607e7696f47812407344704ee649b30ea4d64ab56779" gracePeriod=30
Jul 16 11:35:22.699627 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:22.699580    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-bb5ebc-openshift-default-7c4d969fc5-958jr"
Jul 16 11:35:22.769470 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:22.769362    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d86cfcb1-6d56-4036-968f-f517e9d0d578-istio-envoy\") pod \"d86cfcb1-6d56-4036-968f-f517e9d0d578\" (UID: \"d86cfcb1-6d56-4036-968f-f517e9d0d578\") "
Jul 16 11:35:22.769470 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:22.769410    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d86cfcb1-6d56-4036-968f-f517e9d0d578-workload-socket\") pod \"d86cfcb1-6d56-4036-968f-f517e9d0d578\" (UID: \"d86cfcb1-6d56-4036-968f-f517e9d0d578\") "
Jul 16 11:35:22.769470 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:22.769430    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ss8zs\" (UniqueName: \"kubernetes.io/projected/d86cfcb1-6d56-4036-968f-f517e9d0d578-kube-api-access-ss8zs\") pod \"d86cfcb1-6d56-4036-968f-f517e9d0d578\" (UID: \"d86cfcb1-6d56-4036-968f-f517e9d0d578\") "
Jul 16 11:35:22.769470 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:22.769448    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d86cfcb1-6d56-4036-968f-f517e9d0d578-istio-podinfo\") pod \"d86cfcb1-6d56-4036-968f-f517e9d0d578\" (UID: \"d86cfcb1-6d56-4036-968f-f517e9d0d578\") "
Jul 16 11:35:22.769846 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:22.769486    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d86cfcb1-6d56-4036-968f-f517e9d0d578-workload-certs\") pod \"d86cfcb1-6d56-4036-968f-f517e9d0d578\" (UID: \"d86cfcb1-6d56-4036-968f-f517e9d0d578\") "
Jul 16 11:35:22.769846 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:22.769502    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d86cfcb1-6d56-4036-968f-f517e9d0d578-istiod-ca-cert\") pod \"d86cfcb1-6d56-4036-968f-f517e9d0d578\" (UID: \"d86cfcb1-6d56-4036-968f-f517e9d0d578\") "
Jul 16 11:35:22.769846 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:22.769573    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d86cfcb1-6d56-4036-968f-f517e9d0d578-istio-data\") pod \"d86cfcb1-6d56-4036-968f-f517e9d0d578\" (UID: \"d86cfcb1-6d56-4036-968f-f517e9d0d578\") "
Jul 16 11:35:22.769846 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:22.769655    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d86cfcb1-6d56-4036-968f-f517e9d0d578-istio-token\") pod \"d86cfcb1-6d56-4036-968f-f517e9d0d578\" (UID: \"d86cfcb1-6d56-4036-968f-f517e9d0d578\") "
Jul 16 11:35:22.769846 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:22.769706    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d86cfcb1-6d56-4036-968f-f517e9d0d578-credential-socket\") pod \"d86cfcb1-6d56-4036-968f-f517e9d0d578\" (UID: \"d86cfcb1-6d56-4036-968f-f517e9d0d578\") "
Jul 16 11:35:22.769846 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:22.769702    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d86cfcb1-6d56-4036-968f-f517e9d0d578-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d86cfcb1-6d56-4036-968f-f517e9d0d578" (UID: "d86cfcb1-6d56-4036-968f-f517e9d0d578"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 11:35:22.769846 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:22.769845    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d86cfcb1-6d56-4036-968f-f517e9d0d578-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d86cfcb1-6d56-4036-968f-f517e9d0d578" (UID: "d86cfcb1-6d56-4036-968f-f517e9d0d578"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 11:35:22.770210 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:22.770028    2583 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d86cfcb1-6d56-4036-968f-f517e9d0d578-workload-certs\") on node \"ip-10-0-129-28.ec2.internal\" DevicePath \"\""
Jul 16 11:35:22.770210 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:22.770047    2583 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d86cfcb1-6d56-4036-968f-f517e9d0d578-workload-socket\") on node \"ip-10-0-129-28.ec2.internal\" DevicePath \"\""
Jul 16 11:35:22.770210 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:22.770162    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d86cfcb1-6d56-4036-968f-f517e9d0d578-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d86cfcb1-6d56-4036-968f-f517e9d0d578" (UID: "d86cfcb1-6d56-4036-968f-f517e9d0d578"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 11:35:22.770363 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:22.770230    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d86cfcb1-6d56-4036-968f-f517e9d0d578-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d86cfcb1-6d56-4036-968f-f517e9d0d578" (UID: "d86cfcb1-6d56-4036-968f-f517e9d0d578"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 11:35:22.770691 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:22.770663    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d86cfcb1-6d56-4036-968f-f517e9d0d578-istio-data" (OuterVolumeSpecName: "istio-data") pod "d86cfcb1-6d56-4036-968f-f517e9d0d578" (UID: "d86cfcb1-6d56-4036-968f-f517e9d0d578"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 11:35:22.771914 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:22.771890    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d86cfcb1-6d56-4036-968f-f517e9d0d578-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d86cfcb1-6d56-4036-968f-f517e9d0d578" (UID: "d86cfcb1-6d56-4036-968f-f517e9d0d578"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 11:35:22.771914 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:22.771901    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d86cfcb1-6d56-4036-968f-f517e9d0d578-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d86cfcb1-6d56-4036-968f-f517e9d0d578" (UID: "d86cfcb1-6d56-4036-968f-f517e9d0d578"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 16 11:35:22.772085 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:22.771911    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d86cfcb1-6d56-4036-968f-f517e9d0d578-kube-api-access-ss8zs" (OuterVolumeSpecName: "kube-api-access-ss8zs") pod "d86cfcb1-6d56-4036-968f-f517e9d0d578" (UID: "d86cfcb1-6d56-4036-968f-f517e9d0d578"). InnerVolumeSpecName "kube-api-access-ss8zs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 11:35:22.772085 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:22.771924    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d86cfcb1-6d56-4036-968f-f517e9d0d578-istio-token" (OuterVolumeSpecName: "istio-token") pod "d86cfcb1-6d56-4036-968f-f517e9d0d578" (UID: "d86cfcb1-6d56-4036-968f-f517e9d0d578"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 11:35:22.871018 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:22.870982    2583 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d86cfcb1-6d56-4036-968f-f517e9d0d578-istio-envoy\") on node \"ip-10-0-129-28.ec2.internal\" DevicePath \"\""
Jul 16 11:35:22.871018 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:22.871015    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ss8zs\" (UniqueName: \"kubernetes.io/projected/d86cfcb1-6d56-4036-968f-f517e9d0d578-kube-api-access-ss8zs\") on node \"ip-10-0-129-28.ec2.internal\" DevicePath \"\""
Jul 16 11:35:22.871018 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:22.871024    2583 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d86cfcb1-6d56-4036-968f-f517e9d0d578-istio-podinfo\") on node \"ip-10-0-129-28.ec2.internal\" DevicePath \"\""
Jul 16 11:35:22.871263 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:22.871032    2583 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d86cfcb1-6d56-4036-968f-f517e9d0d578-istiod-ca-cert\") on node \"ip-10-0-129-28.ec2.internal\" DevicePath \"\""
Jul 16 11:35:22.871263 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:22.871040    2583 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d86cfcb1-6d56-4036-968f-f517e9d0d578-istio-data\") on node \"ip-10-0-129-28.ec2.internal\" DevicePath \"\""
Jul 16 11:35:22.871263 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:22.871050    2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d86cfcb1-6d56-4036-968f-f517e9d0d578-istio-token\") on node \"ip-10-0-129-28.ec2.internal\" DevicePath \"\""
Jul 16 11:35:22.871263 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:22.871058    2583 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d86cfcb1-6d56-4036-968f-f517e9d0d578-credential-socket\") on node \"ip-10-0-129-28.ec2.internal\" DevicePath \"\""
Jul 16 11:35:23.543340 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:23.543238    2583 generic.go:356] "Generic (PLEG): container finished" podID="d86cfcb1-6d56-4036-968f-f517e9d0d578" containerID="54c404e072934a9f6f68607e7696f47812407344704ee649b30ea4d64ab56779" exitCode=0
Jul 16 11:35:23.543340 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:23.543276    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-bb5ebc-openshift-default-7c4d969fc5-958jr" event={"ID":"d86cfcb1-6d56-4036-968f-f517e9d0d578","Type":"ContainerDied","Data":"54c404e072934a9f6f68607e7696f47812407344704ee649b30ea4d64ab56779"}
Jul 16 11:35:23.543340 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:23.543290    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-bb5ebc-openshift-default-7c4d969fc5-958jr"
Jul 16 11:35:23.543340 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:23.543332    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-bb5ebc-openshift-default-7c4d969fc5-958jr" event={"ID":"d86cfcb1-6d56-4036-968f-f517e9d0d578","Type":"ContainerDied","Data":"d468626e7d0ea9c998f85f99d63da3065773eb2c013123c5478dc1678b01237a"}
Jul 16 11:35:23.543691 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:23.543355    2583 scope.go:117] "RemoveContainer" containerID="54c404e072934a9f6f68607e7696f47812407344704ee649b30ea4d64ab56779"
Jul 16 11:35:23.552383 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:23.552359    2583 scope.go:117] "RemoveContainer" containerID="597a9426b78dcb0c21641de8b2e8ac132c46e852d4e32df0c9ae619cb015d8a5"
Jul 16 11:35:23.560435 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:23.560287    2583 scope.go:117] "RemoveContainer" containerID="54c404e072934a9f6f68607e7696f47812407344704ee649b30ea4d64ab56779"
Jul 16 11:35:23.560583 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:35:23.560566    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"54c404e072934a9f6f68607e7696f47812407344704ee649b30ea4d64ab56779\": container with ID starting with 54c404e072934a9f6f68607e7696f47812407344704ee649b30ea4d64ab56779 not found: ID does not exist" containerID="54c404e072934a9f6f68607e7696f47812407344704ee649b30ea4d64ab56779"
Jul 16 11:35:23.560649 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:23.560606    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"54c404e072934a9f6f68607e7696f47812407344704ee649b30ea4d64ab56779"} err="failed to get container status \"54c404e072934a9f6f68607e7696f47812407344704ee649b30ea4d64ab56779\": rpc error: code = NotFound desc = could not find container \"54c404e072934a9f6f68607e7696f47812407344704ee649b30ea4d64ab56779\": container with ID starting with 54c404e072934a9f6f68607e7696f47812407344704ee649b30ea4d64ab56779 not found: ID does not exist"
Jul 16 11:35:23.560649 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:23.560631    2583 scope.go:117] "RemoveContainer" containerID="597a9426b78dcb0c21641de8b2e8ac132c46e852d4e32df0c9ae619cb015d8a5"
Jul 16 11:35:23.560919 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:35:23.560891    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"597a9426b78dcb0c21641de8b2e8ac132c46e852d4e32df0c9ae619cb015d8a5\": container with ID starting with 597a9426b78dcb0c21641de8b2e8ac132c46e852d4e32df0c9ae619cb015d8a5 not found: ID does not exist" containerID="597a9426b78dcb0c21641de8b2e8ac132c46e852d4e32df0c9ae619cb015d8a5"
Jul 16 11:35:23.560974 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:23.560928    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"597a9426b78dcb0c21641de8b2e8ac132c46e852d4e32df0c9ae619cb015d8a5"} err="failed to get container status \"597a9426b78dcb0c21641de8b2e8ac132c46e852d4e32df0c9ae619cb015d8a5\": rpc error: code = NotFound desc = could not find container \"597a9426b78dcb0c21641de8b2e8ac132c46e852d4e32df0c9ae619cb015d8a5\": container with ID starting with 597a9426b78dcb0c21641de8b2e8ac132c46e852d4e32df0c9ae619cb015d8a5 not found: ID does not exist"
Jul 16 11:35:23.576174 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:23.576142    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-bb5ebc-openshift-default-7c4d969fc5-958jr"]
Jul 16 11:35:23.586960 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:23.586929    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-bb5ebc-openshift-default-7c4d969fc5-958jr"]
Jul 16 11:35:24.241142 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:24.241111    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-96e676-bdcd69b49-7jmxp"]
Jul 16 11:35:24.241556 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:24.241324    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-96e676-bdcd69b49-7jmxp" podUID="e119146e-22e0-4d10-8bb9-5dd566b033e8" containerName="maas-api" containerID="cri-o://f1a13a5b396a64b1b71bcc24057cf4a625bfc680f9377cb9f65c006363c0d908" gracePeriod=30
Jul 16 11:35:24.494639 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:24.494561    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-96e676-bdcd69b49-7jmxp"
Jul 16 11:35:24.550931 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:24.550898    2583 generic.go:356] "Generic (PLEG): container finished" podID="e119146e-22e0-4d10-8bb9-5dd566b033e8" containerID="f1a13a5b396a64b1b71bcc24057cf4a625bfc680f9377cb9f65c006363c0d908" exitCode=0
Jul 16 11:35:24.551142 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:24.550947    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-96e676-bdcd69b49-7jmxp"
Jul 16 11:35:24.551142 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:24.550949    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-96e676-bdcd69b49-7jmxp" event={"ID":"e119146e-22e0-4d10-8bb9-5dd566b033e8","Type":"ContainerDied","Data":"f1a13a5b396a64b1b71bcc24057cf4a625bfc680f9377cb9f65c006363c0d908"}
Jul 16 11:35:24.551142 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:24.551087    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-96e676-bdcd69b49-7jmxp" event={"ID":"e119146e-22e0-4d10-8bb9-5dd566b033e8","Type":"ContainerDied","Data":"f45e09e372738604ea3a0ea5ff022faf0e0e2dfaa54f07dfaa69e585eb1e32db"}
Jul 16 11:35:24.551142 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:24.551116    2583 scope.go:117] "RemoveContainer" containerID="f1a13a5b396a64b1b71bcc24057cf4a625bfc680f9377cb9f65c006363c0d908"
Jul 16 11:35:24.559965 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:24.559944    2583 scope.go:117] "RemoveContainer" containerID="f1a13a5b396a64b1b71bcc24057cf4a625bfc680f9377cb9f65c006363c0d908"
Jul 16 11:35:24.560292 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:35:24.560275    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f1a13a5b396a64b1b71bcc24057cf4a625bfc680f9377cb9f65c006363c0d908\": container with ID starting with f1a13a5b396a64b1b71bcc24057cf4a625bfc680f9377cb9f65c006363c0d908 not found: ID does not exist" containerID="f1a13a5b396a64b1b71bcc24057cf4a625bfc680f9377cb9f65c006363c0d908"
Jul 16 11:35:24.560358 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:24.560304    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f1a13a5b396a64b1b71bcc24057cf4a625bfc680f9377cb9f65c006363c0d908"} err="failed to get container status \"f1a13a5b396a64b1b71bcc24057cf4a625bfc680f9377cb9f65c006363c0d908\": rpc error: code = NotFound desc = could not find container \"f1a13a5b396a64b1b71bcc24057cf4a625bfc680f9377cb9f65c006363c0d908\": container with ID starting with f1a13a5b396a64b1b71bcc24057cf4a625bfc680f9377cb9f65c006363c0d908 not found: ID does not exist"
Jul 16 11:35:24.587199 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:24.587157    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e119146e-22e0-4d10-8bb9-5dd566b033e8-openshift-service-ca\") pod \"e119146e-22e0-4d10-8bb9-5dd566b033e8\" (UID: \"e119146e-22e0-4d10-8bb9-5dd566b033e8\") "
Jul 16 11:35:24.587343 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:24.587228    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dz682\" (UniqueName: \"kubernetes.io/projected/e119146e-22e0-4d10-8bb9-5dd566b033e8-kube-api-access-dz682\") pod \"e119146e-22e0-4d10-8bb9-5dd566b033e8\" (UID: \"e119146e-22e0-4d10-8bb9-5dd566b033e8\") "
Jul 16 11:35:24.587343 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:24.587299    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e119146e-22e0-4d10-8bb9-5dd566b033e8-maas-api-tls\") pod \"e119146e-22e0-4d10-8bb9-5dd566b033e8\" (UID: \"e119146e-22e0-4d10-8bb9-5dd566b033e8\") "
Jul 16 11:35:24.587576 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:24.587551    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e119146e-22e0-4d10-8bb9-5dd566b033e8-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e119146e-22e0-4d10-8bb9-5dd566b033e8" (UID: "e119146e-22e0-4d10-8bb9-5dd566b033e8"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 11:35:24.589350 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:24.589318    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e119146e-22e0-4d10-8bb9-5dd566b033e8-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e119146e-22e0-4d10-8bb9-5dd566b033e8" (UID: "e119146e-22e0-4d10-8bb9-5dd566b033e8"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 11:35:24.589437 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:24.589352    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e119146e-22e0-4d10-8bb9-5dd566b033e8-kube-api-access-dz682" (OuterVolumeSpecName: "kube-api-access-dz682") pod "e119146e-22e0-4d10-8bb9-5dd566b033e8" (UID: "e119146e-22e0-4d10-8bb9-5dd566b033e8"). InnerVolumeSpecName "kube-api-access-dz682". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 11:35:24.688095 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:24.688061    2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e119146e-22e0-4d10-8bb9-5dd566b033e8-openshift-service-ca\") on node \"ip-10-0-129-28.ec2.internal\" DevicePath \"\""
Jul 16 11:35:24.688095 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:24.688087    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dz682\" (UniqueName: \"kubernetes.io/projected/e119146e-22e0-4d10-8bb9-5dd566b033e8-kube-api-access-dz682\") on node \"ip-10-0-129-28.ec2.internal\" DevicePath \"\""
Jul 16 11:35:24.688095 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:24.688096    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e119146e-22e0-4d10-8bb9-5dd566b033e8-maas-api-tls\") on node \"ip-10-0-129-28.ec2.internal\" DevicePath \"\""
Jul 16 11:35:24.876473 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:24.876435    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-96e676-bdcd69b49-7jmxp"]
Jul 16 11:35:24.878881 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:24.878852    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-96e676-bdcd69b49-7jmxp"]
Jul 16 11:35:25.228435 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:25.228399    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d86cfcb1-6d56-4036-968f-f517e9d0d578" path="/var/lib/kubelet/pods/d86cfcb1-6d56-4036-968f-f517e9d0d578/volumes"
Jul 16 11:35:25.228821 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:25.228808    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e119146e-22e0-4d10-8bb9-5dd566b033e8" path="/var/lib/kubelet/pods/e119146e-22e0-4d10-8bb9-5dd566b033e8/volumes"
Jul 16 11:35:32.129237 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:32.129205    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-c71f6c-openshift-default-579f48d9c9-h7dvl"]
Jul 16 11:35:32.129759 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:32.129739    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2890a09f-e805-4f7d-8752-6cdc7b826206" containerName="maas-api"
Jul 16 11:35:32.129820 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:32.129763    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="2890a09f-e805-4f7d-8752-6cdc7b826206" containerName="maas-api"
Jul 16 11:35:32.129820 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:32.129786    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d86cfcb1-6d56-4036-968f-f517e9d0d578" containerName="istio-proxy"
Jul 16 11:35:32.129820 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:32.129795    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="d86cfcb1-6d56-4036-968f-f517e9d0d578" containerName="istio-proxy"
Jul 16 11:35:32.129820 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:32.129807    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e119146e-22e0-4d10-8bb9-5dd566b033e8" containerName="maas-api"
Jul 16 11:35:32.129820 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:32.129816    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="e119146e-22e0-4d10-8bb9-5dd566b033e8" containerName="maas-api"
Jul 16 11:35:32.129992 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:32.129848    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d86cfcb1-6d56-4036-968f-f517e9d0d578" containerName="istio-proxy"
Jul 16 11:35:32.129992 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:32.129858    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="d86cfcb1-6d56-4036-968f-f517e9d0d578" containerName="istio-proxy"
Jul 16 11:35:32.129992 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:32.129936    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="e119146e-22e0-4d10-8bb9-5dd566b033e8" containerName="maas-api"
Jul 16 11:35:32.129992 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:32.129948    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="d86cfcb1-6d56-4036-968f-f517e9d0d578" containerName="istio-proxy"
Jul 16 11:35:32.129992 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:32.129963    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="2890a09f-e805-4f7d-8752-6cdc7b826206" containerName="maas-api"
Jul 16 11:35:32.132338 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:32.132320    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-c71f6c-openshift-default-579f48d9c9-h7dvl"
Jul 16 11:35:32.134800 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:32.134774    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-c71f6c-openshift-default-dockercfg-ps6p7\""
Jul 16 11:35:32.146049 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:32.146020    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-c71f6c-openshift-default-579f48d9c9-h7dvl"]
Jul 16 11:35:32.259045 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:32.259001    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/37cb79f1-ba1b-455b-988c-317aa499edbb-workload-socket\") pod \"e2e-api-a-c71f6c-openshift-default-579f48d9c9-h7dvl\" (UID: \"37cb79f1-ba1b-455b-988c-317aa499edbb\") " pod="openshift-ingress/e2e-api-a-c71f6c-openshift-default-579f48d9c9-h7dvl"
Jul 16 11:35:32.259229 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:32.259050    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/37cb79f1-ba1b-455b-988c-317aa499edbb-istio-envoy\") pod \"e2e-api-a-c71f6c-openshift-default-579f48d9c9-h7dvl\" (UID: \"37cb79f1-ba1b-455b-988c-317aa499edbb\") " pod="openshift-ingress/e2e-api-a-c71f6c-openshift-default-579f48d9c9-h7dvl"
Jul 16 11:35:32.259229 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:32.259102    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/37cb79f1-ba1b-455b-988c-317aa499edbb-istio-podinfo\") pod \"e2e-api-a-c71f6c-openshift-default-579f48d9c9-h7dvl\" (UID: \"37cb79f1-ba1b-455b-988c-317aa499edbb\") " pod="openshift-ingress/e2e-api-a-c71f6c-openshift-default-579f48d9c9-h7dvl"
Jul 16 11:35:32.259229 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:32.259146    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/37cb79f1-ba1b-455b-988c-317aa499edbb-credential-socket\") pod \"e2e-api-a-c71f6c-openshift-default-579f48d9c9-h7dvl\" (UID: \"37cb79f1-ba1b-455b-988c-317aa499edbb\") " pod="openshift-ingress/e2e-api-a-c71f6c-openshift-default-579f48d9c9-h7dvl"
Jul 16 11:35:32.259229 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:32.259174    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/37cb79f1-ba1b-455b-988c-317aa499edbb-workload-certs\") pod \"e2e-api-a-c71f6c-openshift-default-579f48d9c9-h7dvl\" (UID: \"37cb79f1-ba1b-455b-988c-317aa499edbb\") " pod="openshift-ingress/e2e-api-a-c71f6c-openshift-default-579f48d9c9-h7dvl"
Jul 16 11:35:32.259362 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:32.259253    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/37cb79f1-ba1b-455b-988c-317aa499edbb-istio-token\") pod \"e2e-api-a-c71f6c-openshift-default-579f48d9c9-h7dvl\" (UID: \"37cb79f1-ba1b-455b-988c-317aa499edbb\") " pod="openshift-ingress/e2e-api-a-c71f6c-openshift-default-579f48d9c9-h7dvl"
Jul 16 11:35:32.259362 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:32.259300    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/37cb79f1-ba1b-455b-988c-317aa499edbb-istiod-ca-cert\") pod \"e2e-api-a-c71f6c-openshift-default-579f48d9c9-h7dvl\" (UID: \"37cb79f1-ba1b-455b-988c-317aa499edbb\") " pod="openshift-ingress/e2e-api-a-c71f6c-openshift-default-579f48d9c9-h7dvl"
Jul 16 11:35:32.259362 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:32.259324    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f8wwj\" (UniqueName: \"kubernetes.io/projected/37cb79f1-ba1b-455b-988c-317aa499edbb-kube-api-access-f8wwj\") pod \"e2e-api-a-c71f6c-openshift-default-579f48d9c9-h7dvl\" (UID: \"37cb79f1-ba1b-455b-988c-317aa499edbb\") " pod="openshift-ingress/e2e-api-a-c71f6c-openshift-default-579f48d9c9-h7dvl"
Jul 16 11:35:32.259362 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:32.259344    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/37cb79f1-ba1b-455b-988c-317aa499edbb-istio-data\") pod \"e2e-api-a-c71f6c-openshift-default-579f48d9c9-h7dvl\" (UID: \"37cb79f1-ba1b-455b-988c-317aa499edbb\") " pod="openshift-ingress/e2e-api-a-c71f6c-openshift-default-579f48d9c9-h7dvl"
Jul 16 11:35:32.360141 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:32.360101    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/37cb79f1-ba1b-455b-988c-317aa499edbb-workload-socket\") pod \"e2e-api-a-c71f6c-openshift-default-579f48d9c9-h7dvl\" (UID: \"37cb79f1-ba1b-455b-988c-317aa499edbb\") " pod="openshift-ingress/e2e-api-a-c71f6c-openshift-default-579f48d9c9-h7dvl"
Jul 16 11:35:32.360141 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:32.360140    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/37cb79f1-ba1b-455b-988c-317aa499edbb-istio-envoy\") pod \"e2e-api-a-c71f6c-openshift-default-579f48d9c9-h7dvl\" (UID: \"37cb79f1-ba1b-455b-988c-317aa499edbb\") " pod="openshift-ingress/e2e-api-a-c71f6c-openshift-default-579f48d9c9-h7dvl"
Jul 16 11:35:32.360407 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:32.360167    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/37cb79f1-ba1b-455b-988c-317aa499edbb-istio-podinfo\") pod \"e2e-api-a-c71f6c-openshift-default-579f48d9c9-h7dvl\" (UID: \"37cb79f1-ba1b-455b-988c-317aa499edbb\") " pod="openshift-ingress/e2e-api-a-c71f6c-openshift-default-579f48d9c9-h7dvl"
Jul 16 11:35:32.360407 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:32.360202    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/37cb79f1-ba1b-455b-988c-317aa499edbb-credential-socket\") pod \"e2e-api-a-c71f6c-openshift-default-579f48d9c9-h7dvl\" (UID: \"37cb79f1-ba1b-455b-988c-317aa499edbb\") " pod="openshift-ingress/e2e-api-a-c71f6c-openshift-default-579f48d9c9-h7dvl"
Jul 16 11:35:32.360407 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:32.360226    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/37cb79f1-ba1b-455b-988c-317aa499edbb-workload-certs\") pod \"e2e-api-a-c71f6c-openshift-default-579f48d9c9-h7dvl\" (UID: \"37cb79f1-ba1b-455b-988c-317aa499edbb\") " pod="openshift-ingress/e2e-api-a-c71f6c-openshift-default-579f48d9c9-h7dvl"
Jul 16 11:35:32.360730 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:32.360703    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/37cb79f1-ba1b-455b-988c-317aa499edbb-workload-socket\") pod \"e2e-api-a-c71f6c-openshift-default-579f48d9c9-h7dvl\" (UID: \"37cb79f1-ba1b-455b-988c-317aa499edbb\") " pod="openshift-ingress/e2e-api-a-c71f6c-openshift-default-579f48d9c9-h7dvl"
Jul 16 11:35:32.360918 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:32.360722    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/37cb79f1-ba1b-455b-988c-317aa499edbb-istio-token\") pod \"e2e-api-a-c71f6c-openshift-default-579f48d9c9-h7dvl\" (UID: \"37cb79f1-ba1b-455b-988c-317aa499edbb\") " pod="openshift-ingress/e2e-api-a-c71f6c-openshift-default-579f48d9c9-h7dvl"
Jul 16 11:35:32.361067 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:32.360943    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/37cb79f1-ba1b-455b-988c-317aa499edbb-istiod-ca-cert\") pod \"e2e-api-a-c71f6c-openshift-default-579f48d9c9-h7dvl\" (UID: \"37cb79f1-ba1b-455b-988c-317aa499edbb\") " pod="openshift-ingress/e2e-api-a-c71f6c-openshift-default-579f48d9c9-h7dvl"
Jul 16 11:35:32.361067 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:32.360749    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/37cb79f1-ba1b-455b-988c-317aa499edbb-credential-socket\") pod \"e2e-api-a-c71f6c-openshift-default-579f48d9c9-h7dvl\" (UID: \"37cb79f1-ba1b-455b-988c-317aa499edbb\") " pod="openshift-ingress/e2e-api-a-c71f6c-openshift-default-579f48d9c9-h7dvl"
Jul 16 11:35:32.361067 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:32.360979    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f8wwj\" (UniqueName: \"kubernetes.io/projected/37cb79f1-ba1b-455b-988c-317aa499edbb-kube-api-access-f8wwj\") pod \"e2e-api-a-c71f6c-openshift-default-579f48d9c9-h7dvl\" (UID: \"37cb79f1-ba1b-455b-988c-317aa499edbb\") " pod="openshift-ingress/e2e-api-a-c71f6c-openshift-default-579f48d9c9-h7dvl"
Jul 16 11:35:32.361067 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:32.360760    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/37cb79f1-ba1b-455b-988c-317aa499edbb-workload-certs\") pod \"e2e-api-a-c71f6c-openshift-default-579f48d9c9-h7dvl\" (UID: \"37cb79f1-ba1b-455b-988c-317aa499edbb\") " pod="openshift-ingress/e2e-api-a-c71f6c-openshift-default-579f48d9c9-h7dvl"
Jul 16 11:35:32.361299 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:32.361084    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/37cb79f1-ba1b-455b-988c-317aa499edbb-istio-data\") pod \"e2e-api-a-c71f6c-openshift-default-579f48d9c9-h7dvl\" (UID: \"37cb79f1-ba1b-455b-988c-317aa499edbb\") " pod="openshift-ingress/e2e-api-a-c71f6c-openshift-default-579f48d9c9-h7dvl"
Jul 16 11:35:32.361424 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:32.361401    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/37cb79f1-ba1b-455b-988c-317aa499edbb-istio-data\") pod \"e2e-api-a-c71f6c-openshift-default-579f48d9c9-h7dvl\" (UID: \"37cb79f1-ba1b-455b-988c-317aa499edbb\") " pod="openshift-ingress/e2e-api-a-c71f6c-openshift-default-579f48d9c9-h7dvl"
Jul 16 11:35:32.361631 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:32.361610    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/37cb79f1-ba1b-455b-988c-317aa499edbb-istiod-ca-cert\") pod \"e2e-api-a-c71f6c-openshift-default-579f48d9c9-h7dvl\" (UID: \"37cb79f1-ba1b-455b-988c-317aa499edbb\") " pod="openshift-ingress/e2e-api-a-c71f6c-openshift-default-579f48d9c9-h7dvl"
Jul 16 11:35:32.362655 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:32.362630    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/37cb79f1-ba1b-455b-988c-317aa499edbb-istio-envoy\") pod \"e2e-api-a-c71f6c-openshift-default-579f48d9c9-h7dvl\" (UID: \"37cb79f1-ba1b-455b-988c-317aa499edbb\") " pod="openshift-ingress/e2e-api-a-c71f6c-openshift-default-579f48d9c9-h7dvl"
Jul 16 11:35:32.362895 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:32.362876    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/37cb79f1-ba1b-455b-988c-317aa499edbb-istio-podinfo\") pod \"e2e-api-a-c71f6c-openshift-default-579f48d9c9-h7dvl\" (UID: \"37cb79f1-ba1b-455b-988c-317aa499edbb\") " pod="openshift-ingress/e2e-api-a-c71f6c-openshift-default-579f48d9c9-h7dvl"
Jul 16 11:35:32.371461 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:32.371434    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/37cb79f1-ba1b-455b-988c-317aa499edbb-istio-token\") pod \"e2e-api-a-c71f6c-openshift-default-579f48d9c9-h7dvl\" (UID: \"37cb79f1-ba1b-455b-988c-317aa499edbb\") " pod="openshift-ingress/e2e-api-a-c71f6c-openshift-default-579f48d9c9-h7dvl"
Jul 16 11:35:32.371757 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:32.371739    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f8wwj\" (UniqueName: \"kubernetes.io/projected/37cb79f1-ba1b-455b-988c-317aa499edbb-kube-api-access-f8wwj\") pod \"e2e-api-a-c71f6c-openshift-default-579f48d9c9-h7dvl\" (UID: \"37cb79f1-ba1b-455b-988c-317aa499edbb\") " pod="openshift-ingress/e2e-api-a-c71f6c-openshift-default-579f48d9c9-h7dvl"
Jul 16 11:35:32.442989 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:32.442936    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-c71f6c-openshift-default-579f48d9c9-h7dvl"
Jul 16 11:35:32.586254 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:32.586230    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-c71f6c-openshift-default-579f48d9c9-h7dvl"]
Jul 16 11:35:32.588866 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:35:32.588838    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37cb79f1_ba1b_455b_988c_317aa499edbb.slice/crio-2972dca12c5934a7c0e8e311f610ae5db7995e41fb0063844ded31e31c926c99 WatchSource:0}: Error finding container 2972dca12c5934a7c0e8e311f610ae5db7995e41fb0063844ded31e31c926c99: Status 404 returned error can't find the container with id 2972dca12c5934a7c0e8e311f610ae5db7995e41fb0063844ded31e31c926c99
Jul 16 11:35:32.590938 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:32.590904    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 11:35:32.591038 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:32.590972    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 11:35:32.591038 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:32.591004    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 11:35:33.594484 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:33.594444    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-c71f6c-openshift-default-579f48d9c9-h7dvl" event={"ID":"37cb79f1-ba1b-455b-988c-317aa499edbb","Type":"ContainerStarted","Data":"f533ea2a0ea9da98669e57d6fbecfdab6e6a89fa4abf00b7c9260baf0bc940b3"}
Jul 16 11:35:33.594904 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:33.594504    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-c71f6c-openshift-default-579f48d9c9-h7dvl" event={"ID":"37cb79f1-ba1b-455b-988c-317aa499edbb","Type":"ContainerStarted","Data":"2972dca12c5934a7c0e8e311f610ae5db7995e41fb0063844ded31e31c926c99"}
Jul 16 11:35:33.616473 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:33.616424    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-c71f6c-openshift-default-579f48d9c9-h7dvl" podStartSLOduration=1.616409424 podStartE2EDuration="1.616409424s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 11:35:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 11:35:33.613748697 +0000 UTC m=+1926.247970151" watchObservedRunningTime="2026-07-16 11:35:33.616409424 +0000 UTC m=+1926.250630878"
Jul 16 11:35:34.444043 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:34.444001    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-c71f6c-openshift-default-579f48d9c9-h7dvl"
Jul 16 11:35:34.449109 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:34.449080    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-c71f6c-openshift-default-579f48d9c9-h7dvl"
Jul 16 11:35:34.599467 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:34.599434    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-c71f6c-openshift-default-579f48d9c9-h7dvl"
Jul 16 11:35:34.600516 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:34.600495    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-c71f6c-openshift-default-579f48d9c9-h7dvl"
Jul 16 11:35:34.727044 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:34.726962    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-c71f6c-6bb9bb88d-cnnpq"]
Jul 16 11:35:34.727615 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:34.727577    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="d86cfcb1-6d56-4036-968f-f517e9d0d578" containerName="istio-proxy"
Jul 16 11:35:34.730007 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:34.729984    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c71f6c-6bb9bb88d-cnnpq"
Jul 16 11:35:34.732905 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:34.732878    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-c71f6c-serving-cert\""
Jul 16 11:35:34.743991 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:34.743958    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-c71f6c-6bb9bb88d-cnnpq"]
Jul 16 11:35:34.883212 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:34.883172    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f5f38b5-5e0c-4fff-b6cd-ce752c75bd9a-openshift-service-ca\") pod \"maas-api-e2e-api-a-c71f6c-6bb9bb88d-cnnpq\" (UID: \"6f5f38b5-5e0c-4fff-b6cd-ce752c75bd9a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c71f6c-6bb9bb88d-cnnpq"
Jul 16 11:35:34.883212 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:34.883213    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-44qsn\" (UniqueName: \"kubernetes.io/projected/6f5f38b5-5e0c-4fff-b6cd-ce752c75bd9a-kube-api-access-44qsn\") pod \"maas-api-e2e-api-a-c71f6c-6bb9bb88d-cnnpq\" (UID: \"6f5f38b5-5e0c-4fff-b6cd-ce752c75bd9a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c71f6c-6bb9bb88d-cnnpq"
Jul 16 11:35:34.883437 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:34.883239    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6f5f38b5-5e0c-4fff-b6cd-ce752c75bd9a-maas-api-tls\") pod \"maas-api-e2e-api-a-c71f6c-6bb9bb88d-cnnpq\" (UID: \"6f5f38b5-5e0c-4fff-b6cd-ce752c75bd9a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c71f6c-6bb9bb88d-cnnpq"
Jul 16 11:35:34.984104 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:34.983995    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f5f38b5-5e0c-4fff-b6cd-ce752c75bd9a-openshift-service-ca\") pod \"maas-api-e2e-api-a-c71f6c-6bb9bb88d-cnnpq\" (UID: \"6f5f38b5-5e0c-4fff-b6cd-ce752c75bd9a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c71f6c-6bb9bb88d-cnnpq"
Jul 16 11:35:34.984104 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:34.984038    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-44qsn\" (UniqueName: \"kubernetes.io/projected/6f5f38b5-5e0c-4fff-b6cd-ce752c75bd9a-kube-api-access-44qsn\") pod \"maas-api-e2e-api-a-c71f6c-6bb9bb88d-cnnpq\" (UID: \"6f5f38b5-5e0c-4fff-b6cd-ce752c75bd9a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c71f6c-6bb9bb88d-cnnpq"
Jul 16 11:35:34.984104 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:34.984075    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6f5f38b5-5e0c-4fff-b6cd-ce752c75bd9a-maas-api-tls\") pod \"maas-api-e2e-api-a-c71f6c-6bb9bb88d-cnnpq\" (UID: \"6f5f38b5-5e0c-4fff-b6cd-ce752c75bd9a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c71f6c-6bb9bb88d-cnnpq"
Jul 16 11:35:34.984811 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:34.984790    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f5f38b5-5e0c-4fff-b6cd-ce752c75bd9a-openshift-service-ca\") pod \"maas-api-e2e-api-a-c71f6c-6bb9bb88d-cnnpq\" (UID: \"6f5f38b5-5e0c-4fff-b6cd-ce752c75bd9a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c71f6c-6bb9bb88d-cnnpq"
Jul 16 11:35:34.986706 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:34.986687    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6f5f38b5-5e0c-4fff-b6cd-ce752c75bd9a-maas-api-tls\") pod \"maas-api-e2e-api-a-c71f6c-6bb9bb88d-cnnpq\" (UID: \"6f5f38b5-5e0c-4fff-b6cd-ce752c75bd9a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c71f6c-6bb9bb88d-cnnpq"
Jul 16 11:35:34.993977 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:34.993952    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-44qsn\" (UniqueName: \"kubernetes.io/projected/6f5f38b5-5e0c-4fff-b6cd-ce752c75bd9a-kube-api-access-44qsn\") pod \"maas-api-e2e-api-a-c71f6c-6bb9bb88d-cnnpq\" (UID: \"6f5f38b5-5e0c-4fff-b6cd-ce752c75bd9a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c71f6c-6bb9bb88d-cnnpq"
Jul 16 11:35:35.044811 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:35.044763    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c71f6c-6bb9bb88d-cnnpq"
Jul 16 11:35:35.183881 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:35.183855    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-c71f6c-6bb9bb88d-cnnpq"]
Jul 16 11:35:35.186310 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:35:35.186281    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f5f38b5_5e0c_4fff_b6cd_ce752c75bd9a.slice/crio-b78af195589205323a2c75e766e14b183bf643dcea5af8880eeb52a9e7b92bcb WatchSource:0}: Error finding container b78af195589205323a2c75e766e14b183bf643dcea5af8880eeb52a9e7b92bcb: Status 404 returned error can't find the container with id b78af195589205323a2c75e766e14b183bf643dcea5af8880eeb52a9e7b92bcb
Jul 16 11:35:35.187712 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:35.187693    2583 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 16 11:35:35.605922 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:35.605879    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c71f6c-6bb9bb88d-cnnpq" event={"ID":"6f5f38b5-5e0c-4fff-b6cd-ce752c75bd9a","Type":"ContainerStarted","Data":"b78af195589205323a2c75e766e14b183bf643dcea5af8880eeb52a9e7b92bcb"}
Jul 16 11:35:36.612964 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:36.612924    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c71f6c-6bb9bb88d-cnnpq" event={"ID":"6f5f38b5-5e0c-4fff-b6cd-ce752c75bd9a","Type":"ContainerStarted","Data":"abf92cf9c9fb9580aeaf8847db2ca31c87be045f1e58cc7067c91fc2846f71ab"}
Jul 16 11:35:36.613370 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:36.613178    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c71f6c-6bb9bb88d-cnnpq"
Jul 16 11:35:36.631229 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:36.631177    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c71f6c-6bb9bb88d-cnnpq" podStartSLOduration=2.17802768 podStartE2EDuration="2.631161072s" totalImagesPullingTime="453.133392ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 11:35:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 11:35:36.629746213 +0000 UTC m=+1929.263967669" watchObservedRunningTime="2026-07-16 11:35:36.631161072 +0000 UTC m=+1929.265382525"
Jul 16 11:35:42.636891 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:42.635930    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c71f6c-6bb9bb88d-cnnpq"
Jul 16 11:35:43.394721 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:43.394678    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-2aca13-6756d7db4d-msfzw"]
Jul 16 11:35:43.398841 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:43.398818    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2aca13-6756d7db4d-msfzw"
Jul 16 11:35:43.401426 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:43.401401    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-2aca13-serving-cert\""
Jul 16 11:35:43.418231 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:43.418200    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-2aca13-6756d7db4d-msfzw"]
Jul 16 11:35:43.466726 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:43.466692    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bd05fd99-a2cd-4ddb-819f-83b303d06862-maas-api-tls\") pod \"maas-api-e2e-api-b-2aca13-6756d7db4d-msfzw\" (UID: \"bd05fd99-a2cd-4ddb-819f-83b303d06862\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2aca13-6756d7db4d-msfzw"
Jul 16 11:35:43.466726 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:43.466734    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bd05fd99-a2cd-4ddb-819f-83b303d06862-openshift-service-ca\") pod \"maas-api-e2e-api-b-2aca13-6756d7db4d-msfzw\" (UID: \"bd05fd99-a2cd-4ddb-819f-83b303d06862\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2aca13-6756d7db4d-msfzw"
Jul 16 11:35:43.466995 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:43.466786    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5rvhf\" (UniqueName: \"kubernetes.io/projected/bd05fd99-a2cd-4ddb-819f-83b303d06862-kube-api-access-5rvhf\") pod \"maas-api-e2e-api-b-2aca13-6756d7db4d-msfzw\" (UID: \"bd05fd99-a2cd-4ddb-819f-83b303d06862\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2aca13-6756d7db4d-msfzw"
Jul 16 11:35:43.567682 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:43.567635    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bd05fd99-a2cd-4ddb-819f-83b303d06862-openshift-service-ca\") pod \"maas-api-e2e-api-b-2aca13-6756d7db4d-msfzw\" (UID: \"bd05fd99-a2cd-4ddb-819f-83b303d06862\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2aca13-6756d7db4d-msfzw"
Jul 16 11:35:43.567884 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:43.567730    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5rvhf\" (UniqueName: \"kubernetes.io/projected/bd05fd99-a2cd-4ddb-819f-83b303d06862-kube-api-access-5rvhf\") pod \"maas-api-e2e-api-b-2aca13-6756d7db4d-msfzw\" (UID: \"bd05fd99-a2cd-4ddb-819f-83b303d06862\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2aca13-6756d7db4d-msfzw"
Jul 16 11:35:43.567884 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:43.567836    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bd05fd99-a2cd-4ddb-819f-83b303d06862-maas-api-tls\") pod \"maas-api-e2e-api-b-2aca13-6756d7db4d-msfzw\" (UID: \"bd05fd99-a2cd-4ddb-819f-83b303d06862\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2aca13-6756d7db4d-msfzw"
Jul 16 11:35:43.568011 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:35:43.567947    2583 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-2aca13-serving-cert: secret "maas-api-e2e-api-b-2aca13-serving-cert" not found
Jul 16 11:35:43.568067 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:35:43.568011    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bd05fd99-a2cd-4ddb-819f-83b303d06862-maas-api-tls podName:bd05fd99-a2cd-4ddb-819f-83b303d06862 nodeName:}" failed. No retries permitted until 2026-07-16 11:35:44.067990739 +0000 UTC m=+1936.702212172 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/bd05fd99-a2cd-4ddb-819f-83b303d06862-maas-api-tls") pod "maas-api-e2e-api-b-2aca13-6756d7db4d-msfzw" (UID: "bd05fd99-a2cd-4ddb-819f-83b303d06862") : secret "maas-api-e2e-api-b-2aca13-serving-cert" not found
Jul 16 11:35:43.568419 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:43.568393    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bd05fd99-a2cd-4ddb-819f-83b303d06862-openshift-service-ca\") pod \"maas-api-e2e-api-b-2aca13-6756d7db4d-msfzw\" (UID: \"bd05fd99-a2cd-4ddb-819f-83b303d06862\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2aca13-6756d7db4d-msfzw"
Jul 16 11:35:43.581143 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:43.581103    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5rvhf\" (UniqueName: \"kubernetes.io/projected/bd05fd99-a2cd-4ddb-819f-83b303d06862-kube-api-access-5rvhf\") pod \"maas-api-e2e-api-b-2aca13-6756d7db4d-msfzw\" (UID: \"bd05fd99-a2cd-4ddb-819f-83b303d06862\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2aca13-6756d7db4d-msfzw"
Jul 16 11:35:44.073321 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:44.073276    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bd05fd99-a2cd-4ddb-819f-83b303d06862-maas-api-tls\") pod \"maas-api-e2e-api-b-2aca13-6756d7db4d-msfzw\" (UID: \"bd05fd99-a2cd-4ddb-819f-83b303d06862\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2aca13-6756d7db4d-msfzw"
Jul 16 11:35:44.075866 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:44.075838    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bd05fd99-a2cd-4ddb-819f-83b303d06862-maas-api-tls\") pod \"maas-api-e2e-api-b-2aca13-6756d7db4d-msfzw\" (UID: \"bd05fd99-a2cd-4ddb-819f-83b303d06862\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2aca13-6756d7db4d-msfzw"
Jul 16 11:35:44.317049 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:44.317009    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2aca13-6756d7db4d-msfzw"
Jul 16 11:35:44.475685 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:44.475652    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-2aca13-6756d7db4d-msfzw"]
Jul 16 11:35:44.476767 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:35:44.476736    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd05fd99_a2cd_4ddb_819f_83b303d06862.slice/crio-107fee76aee17daaa22713d23ba1c4ce6acc032a60be139db24d8ef3bc60ae84 WatchSource:0}: Error finding container 107fee76aee17daaa22713d23ba1c4ce6acc032a60be139db24d8ef3bc60ae84: Status 404 returned error can't find the container with id 107fee76aee17daaa22713d23ba1c4ce6acc032a60be139db24d8ef3bc60ae84
Jul 16 11:35:44.665692 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:44.665633    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2aca13-6756d7db4d-msfzw" event={"ID":"bd05fd99-a2cd-4ddb-819f-83b303d06862","Type":"ContainerStarted","Data":"107fee76aee17daaa22713d23ba1c4ce6acc032a60be139db24d8ef3bc60ae84"}
Jul 16 11:35:45.671677 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:45.671639    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2aca13-6756d7db4d-msfzw" event={"ID":"bd05fd99-a2cd-4ddb-819f-83b303d06862","Type":"ContainerStarted","Data":"b6decc5c25d4f08f219688055d1cd6d7aa2a537e270c5be3096b1c77916e750d"}
Jul 16 11:35:45.673221 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:45.671763    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2aca13-6756d7db4d-msfzw"
Jul 16 11:35:45.694102 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:45.694048    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2aca13-6756d7db4d-msfzw" podStartSLOduration=2.326684722 podStartE2EDuration="2.69403046s" totalImagesPullingTime="367.345738ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 11:35:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 11:35:45.693632542 +0000 UTC m=+1938.327853995" watchObservedRunningTime="2026-07-16 11:35:45.69403046 +0000 UTC m=+1938.328251915"
Jul 16 11:35:46.678103 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:46.678069    2583 generic.go:356] "Generic (PLEG): container finished" podID="37cb79f1-ba1b-455b-988c-317aa499edbb" containerID="f533ea2a0ea9da98669e57d6fbecfdab6e6a89fa4abf00b7c9260baf0bc940b3" exitCode=0
Jul 16 11:35:46.678507 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:46.678151    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-c71f6c-openshift-default-579f48d9c9-h7dvl" event={"ID":"37cb79f1-ba1b-455b-988c-317aa499edbb","Type":"ContainerDied","Data":"f533ea2a0ea9da98669e57d6fbecfdab6e6a89fa4abf00b7c9260baf0bc940b3"}
Jul 16 11:35:46.678672 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:46.678656    2583 scope.go:117] "RemoveContainer" containerID="f533ea2a0ea9da98669e57d6fbecfdab6e6a89fa4abf00b7c9260baf0bc940b3"
Jul 16 11:35:46.679641 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:46.679613    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 11:35:46.679757 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:46.679693    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 11:35:46.679757 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:46.679739    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 11:35:47.685673 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:47.685628    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-c71f6c-openshift-default-579f48d9c9-h7dvl" event={"ID":"37cb79f1-ba1b-455b-988c-317aa499edbb","Type":"ContainerStarted","Data":"274a4c9feff1e39a950a1bfd824466e6cd20757ceb766ba61633707ad134bd88"}
Jul 16 11:35:48.443721 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:48.443678    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-c71f6c-openshift-default-579f48d9c9-h7dvl"
Jul 16 11:35:48.444834 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:48.444801    2583 patch_prober.go:28] interesting pod/e2e-api-a-c71f6c-openshift-default-579f48d9c9-h7dvl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body=
Jul 16 11:35:48.444977 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:48.444855    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-c71f6c-openshift-default-579f48d9c9-h7dvl" podUID="37cb79f1-ba1b-455b-988c-317aa499edbb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused"
Jul 16 11:35:49.443675 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:49.443630    2583 patch_prober.go:28] interesting pod/e2e-api-a-c71f6c-openshift-default-579f48d9c9-h7dvl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body=
Jul 16 11:35:49.444079 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:49.443697    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-c71f6c-openshift-default-579f48d9c9-h7dvl" podUID="37cb79f1-ba1b-455b-988c-317aa499edbb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused"
Jul 16 11:35:49.600361 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:49.600311    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-c71f6c-openshift-default-579f48d9c9-h7dvl"
Jul 16 11:35:50.443992 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:50.443945    2583 patch_prober.go:28] interesting pod/e2e-api-a-c71f6c-openshift-default-579f48d9c9-h7dvl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body=
Jul 16 11:35:50.444396 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:50.444011    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-c71f6c-openshift-default-579f48d9c9-h7dvl" podUID="37cb79f1-ba1b-455b-988c-317aa499edbb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused"
Jul 16 11:35:51.443910 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:51.443865    2583 patch_prober.go:28] interesting pod/e2e-api-a-c71f6c-openshift-default-579f48d9c9-h7dvl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body=
Jul 16 11:35:51.444097 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:51.443941    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-c71f6c-openshift-default-579f48d9c9-h7dvl" podUID="37cb79f1-ba1b-455b-988c-317aa499edbb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused"
Jul 16 11:35:51.684459 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:51.684430    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2aca13-6756d7db4d-msfzw"
Jul 16 11:35:52.447264 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:52.447230    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-c71f6c-openshift-default-579f48d9c9-h7dvl"
Jul 16 11:35:52.448224 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:35:52.448201    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-c71f6c-openshift-default-579f48d9c9-h7dvl"
Jul 16 11:36:02.895287 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:02.895251    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-c71f6c-6bb9bb88d-cnnpq"]
Jul 16 11:36:02.895837 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:02.895458    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c71f6c-6bb9bb88d-cnnpq" podUID="6f5f38b5-5e0c-4fff-b6cd-ce752c75bd9a" containerName="maas-api" containerID="cri-o://abf92cf9c9fb9580aeaf8847db2ca31c87be045f1e58cc7067c91fc2846f71ab" gracePeriod=30
Jul 16 11:36:03.163425 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:03.163398    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c71f6c-6bb9bb88d-cnnpq"
Jul 16 11:36:03.264676 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:03.264636    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6f5f38b5-5e0c-4fff-b6cd-ce752c75bd9a-maas-api-tls\") pod \"6f5f38b5-5e0c-4fff-b6cd-ce752c75bd9a\" (UID: \"6f5f38b5-5e0c-4fff-b6cd-ce752c75bd9a\") "
Jul 16 11:36:03.264891 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:03.264738    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f5f38b5-5e0c-4fff-b6cd-ce752c75bd9a-openshift-service-ca\") pod \"6f5f38b5-5e0c-4fff-b6cd-ce752c75bd9a\" (UID: \"6f5f38b5-5e0c-4fff-b6cd-ce752c75bd9a\") "
Jul 16 11:36:03.264891 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:03.264805    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-44qsn\" (UniqueName: \"kubernetes.io/projected/6f5f38b5-5e0c-4fff-b6cd-ce752c75bd9a-kube-api-access-44qsn\") pod \"6f5f38b5-5e0c-4fff-b6cd-ce752c75bd9a\" (UID: \"6f5f38b5-5e0c-4fff-b6cd-ce752c75bd9a\") "
Jul 16 11:36:03.265130 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:03.265105    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6f5f38b5-5e0c-4fff-b6cd-ce752c75bd9a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "6f5f38b5-5e0c-4fff-b6cd-ce752c75bd9a" (UID: "6f5f38b5-5e0c-4fff-b6cd-ce752c75bd9a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 11:36:03.266908 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:03.266882    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6f5f38b5-5e0c-4fff-b6cd-ce752c75bd9a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "6f5f38b5-5e0c-4fff-b6cd-ce752c75bd9a" (UID: "6f5f38b5-5e0c-4fff-b6cd-ce752c75bd9a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 11:36:03.267044 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:03.267022    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6f5f38b5-5e0c-4fff-b6cd-ce752c75bd9a-kube-api-access-44qsn" (OuterVolumeSpecName: "kube-api-access-44qsn") pod "6f5f38b5-5e0c-4fff-b6cd-ce752c75bd9a" (UID: "6f5f38b5-5e0c-4fff-b6cd-ce752c75bd9a"). InnerVolumeSpecName "kube-api-access-44qsn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 11:36:03.365887 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:03.365835    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6f5f38b5-5e0c-4fff-b6cd-ce752c75bd9a-maas-api-tls\") on node \"ip-10-0-129-28.ec2.internal\" DevicePath \"\""
Jul 16 11:36:03.365887 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:03.365876    2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f5f38b5-5e0c-4fff-b6cd-ce752c75bd9a-openshift-service-ca\") on node \"ip-10-0-129-28.ec2.internal\" DevicePath \"\""
Jul 16 11:36:03.365887 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:03.365890    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-44qsn\" (UniqueName: \"kubernetes.io/projected/6f5f38b5-5e0c-4fff-b6cd-ce752c75bd9a-kube-api-access-44qsn\") on node \"ip-10-0-129-28.ec2.internal\" DevicePath \"\""
Jul 16 11:36:03.773825 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:03.773785    2583 generic.go:356] "Generic (PLEG): container finished" podID="6f5f38b5-5e0c-4fff-b6cd-ce752c75bd9a" containerID="abf92cf9c9fb9580aeaf8847db2ca31c87be045f1e58cc7067c91fc2846f71ab" exitCode=0
Jul 16 11:36:03.774004 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:03.773843    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c71f6c-6bb9bb88d-cnnpq"
Jul 16 11:36:03.774004 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:03.773844    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c71f6c-6bb9bb88d-cnnpq" event={"ID":"6f5f38b5-5e0c-4fff-b6cd-ce752c75bd9a","Type":"ContainerDied","Data":"abf92cf9c9fb9580aeaf8847db2ca31c87be045f1e58cc7067c91fc2846f71ab"}
Jul 16 11:36:03.774004 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:03.773960    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c71f6c-6bb9bb88d-cnnpq" event={"ID":"6f5f38b5-5e0c-4fff-b6cd-ce752c75bd9a","Type":"ContainerDied","Data":"b78af195589205323a2c75e766e14b183bf643dcea5af8880eeb52a9e7b92bcb"}
Jul 16 11:36:03.774004 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:03.773982    2583 scope.go:117] "RemoveContainer" containerID="abf92cf9c9fb9580aeaf8847db2ca31c87be045f1e58cc7067c91fc2846f71ab"
Jul 16 11:36:03.783158 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:03.783134    2583 scope.go:117] "RemoveContainer" containerID="abf92cf9c9fb9580aeaf8847db2ca31c87be045f1e58cc7067c91fc2846f71ab"
Jul 16 11:36:03.783490 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:36:03.783468    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"abf92cf9c9fb9580aeaf8847db2ca31c87be045f1e58cc7067c91fc2846f71ab\": container with ID starting with abf92cf9c9fb9580aeaf8847db2ca31c87be045f1e58cc7067c91fc2846f71ab not found: ID does not exist" containerID="abf92cf9c9fb9580aeaf8847db2ca31c87be045f1e58cc7067c91fc2846f71ab"
Jul 16 11:36:03.783543 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:03.783500    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"abf92cf9c9fb9580aeaf8847db2ca31c87be045f1e58cc7067c91fc2846f71ab"} err="failed to get container status \"abf92cf9c9fb9580aeaf8847db2ca31c87be045f1e58cc7067c91fc2846f71ab\": rpc error: code = NotFound desc = could not find container \"abf92cf9c9fb9580aeaf8847db2ca31c87be045f1e58cc7067c91fc2846f71ab\": container with ID starting with abf92cf9c9fb9580aeaf8847db2ca31c87be045f1e58cc7067c91fc2846f71ab not found: ID does not exist"
Jul 16 11:36:03.797586 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:03.797545    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-c71f6c-6bb9bb88d-cnnpq"]
Jul 16 11:36:03.802404 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:03.802375    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-c71f6c-6bb9bb88d-cnnpq"]
Jul 16 11:36:05.229136 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:05.229096    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6f5f38b5-5e0c-4fff-b6cd-ce752c75bd9a" path="/var/lib/kubelet/pods/6f5f38b5-5e0c-4fff-b6cd-ce752c75bd9a/volumes"
Jul 16 11:36:09.640774 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:09.640741    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-c71f6c-openshift-default-579f48d9c9-h7dvl"]
Jul 16 11:36:09.641302 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:09.641060    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-c71f6c-openshift-default-579f48d9c9-h7dvl" podUID="37cb79f1-ba1b-455b-988c-317aa499edbb" containerName="istio-proxy" containerID="cri-o://274a4c9feff1e39a950a1bfd824466e6cd20757ceb766ba61633707ad134bd88" gracePeriod=30
Jul 16 11:36:14.837348 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:14.837305    2583 generic.go:356] "Generic (PLEG): container finished" podID="37cb79f1-ba1b-455b-988c-317aa499edbb" containerID="274a4c9feff1e39a950a1bfd824466e6cd20757ceb766ba61633707ad134bd88" exitCode=0
Jul 16 11:36:14.837820 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:14.837445    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-c71f6c-openshift-default-579f48d9c9-h7dvl" event={"ID":"37cb79f1-ba1b-455b-988c-317aa499edbb","Type":"ContainerDied","Data":"274a4c9feff1e39a950a1bfd824466e6cd20757ceb766ba61633707ad134bd88"}
Jul 16 11:36:14.837820 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:14.837507    2583 scope.go:117] "RemoveContainer" containerID="f533ea2a0ea9da98669e57d6fbecfdab6e6a89fa4abf00b7c9260baf0bc940b3"
Jul 16 11:36:14.918054 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:14.918026    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-c71f6c-openshift-default-579f48d9c9-h7dvl"
Jul 16 11:36:15.070059 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:15.070025    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/37cb79f1-ba1b-455b-988c-317aa499edbb-istio-envoy\") pod \"37cb79f1-ba1b-455b-988c-317aa499edbb\" (UID: \"37cb79f1-ba1b-455b-988c-317aa499edbb\") "
Jul 16 11:36:15.070250 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:15.070100    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/37cb79f1-ba1b-455b-988c-317aa499edbb-istio-data\") pod \"37cb79f1-ba1b-455b-988c-317aa499edbb\" (UID: \"37cb79f1-ba1b-455b-988c-317aa499edbb\") "
Jul 16 11:36:15.070250 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:15.070160    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/37cb79f1-ba1b-455b-988c-317aa499edbb-workload-socket\") pod \"37cb79f1-ba1b-455b-988c-317aa499edbb\" (UID: \"37cb79f1-ba1b-455b-988c-317aa499edbb\") "
Jul 16 11:36:15.070250 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:15.070189    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/37cb79f1-ba1b-455b-988c-317aa499edbb-istio-podinfo\") pod \"37cb79f1-ba1b-455b-988c-317aa499edbb\" (UID: \"37cb79f1-ba1b-455b-988c-317aa499edbb\") "
Jul 16 11:36:15.070250 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:15.070221    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/37cb79f1-ba1b-455b-988c-317aa499edbb-credential-socket\") pod \"37cb79f1-ba1b-455b-988c-317aa499edbb\" (UID: \"37cb79f1-ba1b-455b-988c-317aa499edbb\") "
Jul 16 11:36:15.070464 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:15.070254    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f8wwj\" (UniqueName: \"kubernetes.io/projected/37cb79f1-ba1b-455b-988c-317aa499edbb-kube-api-access-f8wwj\") pod \"37cb79f1-ba1b-455b-988c-317aa499edbb\" (UID: \"37cb79f1-ba1b-455b-988c-317aa499edbb\") "
Jul 16 11:36:15.070464 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:15.070290    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/37cb79f1-ba1b-455b-988c-317aa499edbb-workload-certs\") pod \"37cb79f1-ba1b-455b-988c-317aa499edbb\" (UID: \"37cb79f1-ba1b-455b-988c-317aa499edbb\") "
Jul 16 11:36:15.070464 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:15.070405    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/37cb79f1-ba1b-455b-988c-317aa499edbb-istiod-ca-cert\") pod \"37cb79f1-ba1b-455b-988c-317aa499edbb\" (UID: \"37cb79f1-ba1b-455b-988c-317aa499edbb\") "
Jul 16 11:36:15.070656 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:15.070467    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/37cb79f1-ba1b-455b-988c-317aa499edbb-istio-token\") pod \"37cb79f1-ba1b-455b-988c-317aa499edbb\" (UID: \"37cb79f1-ba1b-455b-988c-317aa499edbb\") "
Jul 16 11:36:15.070656 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:15.070470    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37cb79f1-ba1b-455b-988c-317aa499edbb-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "37cb79f1-ba1b-455b-988c-317aa499edbb" (UID: "37cb79f1-ba1b-455b-988c-317aa499edbb"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 11:36:15.070656 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:15.070572    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37cb79f1-ba1b-455b-988c-317aa499edbb-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "37cb79f1-ba1b-455b-988c-317aa499edbb" (UID: "37cb79f1-ba1b-455b-988c-317aa499edbb"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 11:36:15.070885 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:15.070865    2583 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/37cb79f1-ba1b-455b-988c-317aa499edbb-workload-certs\") on node \"ip-10-0-129-28.ec2.internal\" DevicePath \"\""
Jul 16 11:36:15.071043 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:15.071022    2583 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/37cb79f1-ba1b-455b-988c-317aa499edbb-workload-socket\") on node \"ip-10-0-129-28.ec2.internal\" DevicePath \"\""
Jul 16 11:36:15.071043 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:15.070872    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/37cb79f1-ba1b-455b-988c-317aa499edbb-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "37cb79f1-ba1b-455b-988c-317aa499edbb" (UID: "37cb79f1-ba1b-455b-988c-317aa499edbb"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 11:36:15.071191 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:15.070879    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37cb79f1-ba1b-455b-988c-317aa499edbb-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "37cb79f1-ba1b-455b-988c-317aa499edbb" (UID: "37cb79f1-ba1b-455b-988c-317aa499edbb"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 11:36:15.071191 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:15.071116    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37cb79f1-ba1b-455b-988c-317aa499edbb-istio-data" (OuterVolumeSpecName: "istio-data") pod "37cb79f1-ba1b-455b-988c-317aa499edbb" (UID: "37cb79f1-ba1b-455b-988c-317aa499edbb"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 11:36:15.072897 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:15.072870    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/37cb79f1-ba1b-455b-988c-317aa499edbb-kube-api-access-f8wwj" (OuterVolumeSpecName: "kube-api-access-f8wwj") pod "37cb79f1-ba1b-455b-988c-317aa499edbb" (UID: "37cb79f1-ba1b-455b-988c-317aa499edbb"). InnerVolumeSpecName "kube-api-access-f8wwj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 11:36:15.072897 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:15.072886    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37cb79f1-ba1b-455b-988c-317aa499edbb-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "37cb79f1-ba1b-455b-988c-317aa499edbb" (UID: "37cb79f1-ba1b-455b-988c-317aa499edbb"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 11:36:15.073051 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:15.073004    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/37cb79f1-ba1b-455b-988c-317aa499edbb-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "37cb79f1-ba1b-455b-988c-317aa499edbb" (UID: "37cb79f1-ba1b-455b-988c-317aa499edbb"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 16 11:36:15.073051 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:15.073007    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/37cb79f1-ba1b-455b-988c-317aa499edbb-istio-token" (OuterVolumeSpecName: "istio-token") pod "37cb79f1-ba1b-455b-988c-317aa499edbb" (UID: "37cb79f1-ba1b-455b-988c-317aa499edbb"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 11:36:15.171733 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:15.171632    2583 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/37cb79f1-ba1b-455b-988c-317aa499edbb-istio-data\") on node \"ip-10-0-129-28.ec2.internal\" DevicePath \"\""
Jul 16 11:36:15.171733 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:15.171666    2583 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/37cb79f1-ba1b-455b-988c-317aa499edbb-istio-podinfo\") on node \"ip-10-0-129-28.ec2.internal\" DevicePath \"\""
Jul 16 11:36:15.171733 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:15.171679    2583 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/37cb79f1-ba1b-455b-988c-317aa499edbb-credential-socket\") on node \"ip-10-0-129-28.ec2.internal\" DevicePath \"\""
Jul 16 11:36:15.171733 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:15.171691    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f8wwj\" (UniqueName: \"kubernetes.io/projected/37cb79f1-ba1b-455b-988c-317aa499edbb-kube-api-access-f8wwj\") on node \"ip-10-0-129-28.ec2.internal\" DevicePath \"\""
Jul 16 11:36:15.171733 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:15.171704    2583 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/37cb79f1-ba1b-455b-988c-317aa499edbb-istiod-ca-cert\") on node \"ip-10-0-129-28.ec2.internal\" DevicePath \"\""
Jul 16 11:36:15.171733 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:15.171715    2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/37cb79f1-ba1b-455b-988c-317aa499edbb-istio-token\") on node \"ip-10-0-129-28.ec2.internal\" DevicePath \"\""
Jul 16 11:36:15.171733 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:15.171725    2583 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/37cb79f1-ba1b-455b-988c-317aa499edbb-istio-envoy\") on node \"ip-10-0-129-28.ec2.internal\" DevicePath \"\""
Jul 16 11:36:15.846504 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:15.846460    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-c71f6c-openshift-default-579f48d9c9-h7dvl" event={"ID":"37cb79f1-ba1b-455b-988c-317aa499edbb","Type":"ContainerDied","Data":"2972dca12c5934a7c0e8e311f610ae5db7995e41fb0063844ded31e31c926c99"}
Jul 16 11:36:15.847109 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:15.847087    2583 scope.go:117] "RemoveContainer" containerID="274a4c9feff1e39a950a1bfd824466e6cd20757ceb766ba61633707ad134bd88"
Jul 16 11:36:15.847438 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:15.847420    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-c71f6c-openshift-default-579f48d9c9-h7dvl"
Jul 16 11:36:15.870136 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:15.870101    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-c71f6c-openshift-default-579f48d9c9-h7dvl"]
Jul 16 11:36:15.873123 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:15.873095    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-c71f6c-openshift-default-579f48d9c9-h7dvl"]
Jul 16 11:36:16.350493 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:16.350460    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-2aca13-6756d7db4d-msfzw"]
Jul 16 11:36:16.350737 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:16.350689    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2aca13-6756d7db4d-msfzw" podUID="bd05fd99-a2cd-4ddb-819f-83b303d06862" containerName="maas-api" containerID="cri-o://b6decc5c25d4f08f219688055d1cd6d7aa2a537e270c5be3096b1c77916e750d" gracePeriod=30
Jul 16 11:36:16.599960 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:16.599935    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2aca13-6756d7db4d-msfzw"
Jul 16 11:36:16.686731 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:16.686696    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bd05fd99-a2cd-4ddb-819f-83b303d06862-maas-api-tls\") pod \"bd05fd99-a2cd-4ddb-819f-83b303d06862\" (UID: \"bd05fd99-a2cd-4ddb-819f-83b303d06862\") "
Jul 16 11:36:16.686943 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:16.686759    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bd05fd99-a2cd-4ddb-819f-83b303d06862-openshift-service-ca\") pod \"bd05fd99-a2cd-4ddb-819f-83b303d06862\" (UID: \"bd05fd99-a2cd-4ddb-819f-83b303d06862\") "
Jul 16 11:36:16.686943 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:16.686905    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5rvhf\" (UniqueName: \"kubernetes.io/projected/bd05fd99-a2cd-4ddb-819f-83b303d06862-kube-api-access-5rvhf\") pod \"bd05fd99-a2cd-4ddb-819f-83b303d06862\" (UID: \"bd05fd99-a2cd-4ddb-819f-83b303d06862\") "
Jul 16 11:36:16.687094 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:16.687071    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bd05fd99-a2cd-4ddb-819f-83b303d06862-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "bd05fd99-a2cd-4ddb-819f-83b303d06862" (UID: "bd05fd99-a2cd-4ddb-819f-83b303d06862"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 11:36:16.687242 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:16.687224    2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bd05fd99-a2cd-4ddb-819f-83b303d06862-openshift-service-ca\") on node \"ip-10-0-129-28.ec2.internal\" DevicePath \"\""
Jul 16 11:36:16.688859 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:16.688835    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bd05fd99-a2cd-4ddb-819f-83b303d06862-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "bd05fd99-a2cd-4ddb-819f-83b303d06862" (UID: "bd05fd99-a2cd-4ddb-819f-83b303d06862"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 11:36:16.688953 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:16.688872    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bd05fd99-a2cd-4ddb-819f-83b303d06862-kube-api-access-5rvhf" (OuterVolumeSpecName: "kube-api-access-5rvhf") pod "bd05fd99-a2cd-4ddb-819f-83b303d06862" (UID: "bd05fd99-a2cd-4ddb-819f-83b303d06862"). InnerVolumeSpecName "kube-api-access-5rvhf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 11:36:16.787790 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:16.787756    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5rvhf\" (UniqueName: \"kubernetes.io/projected/bd05fd99-a2cd-4ddb-819f-83b303d06862-kube-api-access-5rvhf\") on node \"ip-10-0-129-28.ec2.internal\" DevicePath \"\""
Jul 16 11:36:16.787790 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:16.787786    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bd05fd99-a2cd-4ddb-819f-83b303d06862-maas-api-tls\") on node \"ip-10-0-129-28.ec2.internal\" DevicePath \"\""
Jul 16 11:36:16.853282 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:16.853249    2583 generic.go:356] "Generic (PLEG): container finished" podID="bd05fd99-a2cd-4ddb-819f-83b303d06862" containerID="b6decc5c25d4f08f219688055d1cd6d7aa2a537e270c5be3096b1c77916e750d" exitCode=0
Jul 16 11:36:16.853703 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:16.853293    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2aca13-6756d7db4d-msfzw"
Jul 16 11:36:16.853703 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:16.853338    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2aca13-6756d7db4d-msfzw" event={"ID":"bd05fd99-a2cd-4ddb-819f-83b303d06862","Type":"ContainerDied","Data":"b6decc5c25d4f08f219688055d1cd6d7aa2a537e270c5be3096b1c77916e750d"}
Jul 16 11:36:16.853703 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:16.853397    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2aca13-6756d7db4d-msfzw" event={"ID":"bd05fd99-a2cd-4ddb-819f-83b303d06862","Type":"ContainerDied","Data":"107fee76aee17daaa22713d23ba1c4ce6acc032a60be139db24d8ef3bc60ae84"}
Jul 16 11:36:16.853703 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:16.853424    2583 scope.go:117] "RemoveContainer" containerID="b6decc5c25d4f08f219688055d1cd6d7aa2a537e270c5be3096b1c77916e750d"
Jul 16 11:36:16.863008 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:16.862991    2583 scope.go:117] "RemoveContainer" containerID="b6decc5c25d4f08f219688055d1cd6d7aa2a537e270c5be3096b1c77916e750d"
Jul 16 11:36:16.863277 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:36:16.863258    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b6decc5c25d4f08f219688055d1cd6d7aa2a537e270c5be3096b1c77916e750d\": container with ID starting with b6decc5c25d4f08f219688055d1cd6d7aa2a537e270c5be3096b1c77916e750d not found: ID does not exist" containerID="b6decc5c25d4f08f219688055d1cd6d7aa2a537e270c5be3096b1c77916e750d"
Jul 16 11:36:16.863325 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:16.863289    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b6decc5c25d4f08f219688055d1cd6d7aa2a537e270c5be3096b1c77916e750d"} err="failed to get container status \"b6decc5c25d4f08f219688055d1cd6d7aa2a537e270c5be3096b1c77916e750d\": rpc error: code = NotFound desc = could not find container \"b6decc5c25d4f08f219688055d1cd6d7aa2a537e270c5be3096b1c77916e750d\": container with ID starting with b6decc5c25d4f08f219688055d1cd6d7aa2a537e270c5be3096b1c77916e750d not found: ID does not exist"
Jul 16 11:36:16.875417 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:16.875389    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-2aca13-6756d7db4d-msfzw"]
Jul 16 11:36:16.880170 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:16.880145    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-2aca13-6756d7db4d-msfzw"]
Jul 16 11:36:17.228679 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:17.228647    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="37cb79f1-ba1b-455b-988c-317aa499edbb" path="/var/lib/kubelet/pods/37cb79f1-ba1b-455b-988c-317aa499edbb/volumes"
Jul 16 11:36:17.229064 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:17.229051    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bd05fd99-a2cd-4ddb-819f-83b303d06862" path="/var/lib/kubelet/pods/bd05fd99-a2cd-4ddb-819f-83b303d06862/volumes"
Jul 16 11:36:23.783643 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:23.783399    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-8bae7d-openshift-default-7bf676686c-gp2wq"]
Jul 16 11:36:23.784831 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:23.784801    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="37cb79f1-ba1b-455b-988c-317aa499edbb" containerName="istio-proxy"
Jul 16 11:36:23.785010 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:23.784997    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="37cb79f1-ba1b-455b-988c-317aa499edbb" containerName="istio-proxy"
Jul 16 11:36:23.785127 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:23.785116    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="37cb79f1-ba1b-455b-988c-317aa499edbb" containerName="istio-proxy"
Jul 16 11:36:23.785195 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:23.785187    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="37cb79f1-ba1b-455b-988c-317aa499edbb" containerName="istio-proxy"
Jul 16 11:36:23.785286 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:23.785277    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bd05fd99-a2cd-4ddb-819f-83b303d06862" containerName="maas-api"
Jul 16 11:36:23.785349 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:23.785341    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd05fd99-a2cd-4ddb-819f-83b303d06862" containerName="maas-api"
Jul 16 11:36:23.785456 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:23.785447    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6f5f38b5-5e0c-4fff-b6cd-ce752c75bd9a" containerName="maas-api"
Jul 16 11:36:23.785526 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:23.785518    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="6f5f38b5-5e0c-4fff-b6cd-ce752c75bd9a" containerName="maas-api"
Jul 16 11:36:23.785812 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:23.785796    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="6f5f38b5-5e0c-4fff-b6cd-ce752c75bd9a" containerName="maas-api"
Jul 16 11:36:23.785947 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:23.785937    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="bd05fd99-a2cd-4ddb-819f-83b303d06862" containerName="maas-api"
Jul 16 11:36:23.786038 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:23.786028    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="37cb79f1-ba1b-455b-988c-317aa499edbb" containerName="istio-proxy"
Jul 16 11:36:23.786107 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:23.786098    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="37cb79f1-ba1b-455b-988c-317aa499edbb" containerName="istio-proxy"
Jul 16 11:36:23.790160 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:23.790131    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-8bae7d-openshift-default-7bf676686c-gp2wq"
Jul 16 11:36:23.793207 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:23.793185    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-8bae7d-openshift-default-dockercfg-6wz7r\""
Jul 16 11:36:23.806828 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:23.806630    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-8bae7d-openshift-default-7bf676686c-gp2wq"]
Jul 16 11:36:23.953970 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:23.953933    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/076e3a37-3f46-4197-9447-cf2b1ef9414d-workload-certs\") pod \"e2e-shared-a-8bae7d-openshift-default-7bf676686c-gp2wq\" (UID: \"076e3a37-3f46-4197-9447-cf2b1ef9414d\") " pod="openshift-ingress/e2e-shared-a-8bae7d-openshift-default-7bf676686c-gp2wq"
Jul 16 11:36:23.954140 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:23.953977    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/076e3a37-3f46-4197-9447-cf2b1ef9414d-istio-token\") pod \"e2e-shared-a-8bae7d-openshift-default-7bf676686c-gp2wq\" (UID: \"076e3a37-3f46-4197-9447-cf2b1ef9414d\") " pod="openshift-ingress/e2e-shared-a-8bae7d-openshift-default-7bf676686c-gp2wq"
Jul 16 11:36:23.954140 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:23.954005    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/076e3a37-3f46-4197-9447-cf2b1ef9414d-istio-envoy\") pod \"e2e-shared-a-8bae7d-openshift-default-7bf676686c-gp2wq\" (UID: \"076e3a37-3f46-4197-9447-cf2b1ef9414d\") " pod="openshift-ingress/e2e-shared-a-8bae7d-openshift-default-7bf676686c-gp2wq"
Jul 16 11:36:23.954140 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:23.954094    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/076e3a37-3f46-4197-9447-cf2b1ef9414d-istio-data\") pod \"e2e-shared-a-8bae7d-openshift-default-7bf676686c-gp2wq\" (UID: \"076e3a37-3f46-4197-9447-cf2b1ef9414d\") " pod="openshift-ingress/e2e-shared-a-8bae7d-openshift-default-7bf676686c-gp2wq"
Jul 16 11:36:23.954140 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:23.954124    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/076e3a37-3f46-4197-9447-cf2b1ef9414d-istiod-ca-cert\") pod \"e2e-shared-a-8bae7d-openshift-default-7bf676686c-gp2wq\" (UID: \"076e3a37-3f46-4197-9447-cf2b1ef9414d\") " pod="openshift-ingress/e2e-shared-a-8bae7d-openshift-default-7bf676686c-gp2wq"
Jul 16 11:36:23.954357 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:23.954147    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/076e3a37-3f46-4197-9447-cf2b1ef9414d-istio-podinfo\") pod \"e2e-shared-a-8bae7d-openshift-default-7bf676686c-gp2wq\" (UID: \"076e3a37-3f46-4197-9447-cf2b1ef9414d\") " pod="openshift-ingress/e2e-shared-a-8bae7d-openshift-default-7bf676686c-gp2wq"
Jul 16 11:36:23.954357 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:23.954190    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/076e3a37-3f46-4197-9447-cf2b1ef9414d-workload-socket\") pod \"e2e-shared-a-8bae7d-openshift-default-7bf676686c-gp2wq\" (UID: \"076e3a37-3f46-4197-9447-cf2b1ef9414d\") " pod="openshift-ingress/e2e-shared-a-8bae7d-openshift-default-7bf676686c-gp2wq"
Jul 16 11:36:23.954357 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:23.954210    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wltrf\" (UniqueName: \"kubernetes.io/projected/076e3a37-3f46-4197-9447-cf2b1ef9414d-kube-api-access-wltrf\") pod \"e2e-shared-a-8bae7d-openshift-default-7bf676686c-gp2wq\" (UID: \"076e3a37-3f46-4197-9447-cf2b1ef9414d\") " pod="openshift-ingress/e2e-shared-a-8bae7d-openshift-default-7bf676686c-gp2wq"
Jul 16 11:36:23.954357 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:23.954283    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/076e3a37-3f46-4197-9447-cf2b1ef9414d-credential-socket\") pod \"e2e-shared-a-8bae7d-openshift-default-7bf676686c-gp2wq\" (UID: \"076e3a37-3f46-4197-9447-cf2b1ef9414d\") " pod="openshift-ingress/e2e-shared-a-8bae7d-openshift-default-7bf676686c-gp2wq"
Jul 16 11:36:24.055380 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:24.055267    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/076e3a37-3f46-4197-9447-cf2b1ef9414d-istio-envoy\") pod \"e2e-shared-a-8bae7d-openshift-default-7bf676686c-gp2wq\" (UID: \"076e3a37-3f46-4197-9447-cf2b1ef9414d\") " pod="openshift-ingress/e2e-shared-a-8bae7d-openshift-default-7bf676686c-gp2wq"
Jul 16 11:36:24.055380 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:24.055354    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/076e3a37-3f46-4197-9447-cf2b1ef9414d-istio-data\") pod \"e2e-shared-a-8bae7d-openshift-default-7bf676686c-gp2wq\" (UID: \"076e3a37-3f46-4197-9447-cf2b1ef9414d\") " pod="openshift-ingress/e2e-shared-a-8bae7d-openshift-default-7bf676686c-gp2wq"
Jul 16 11:36:24.055380 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:24.055382    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/076e3a37-3f46-4197-9447-cf2b1ef9414d-istiod-ca-cert\") pod \"e2e-shared-a-8bae7d-openshift-default-7bf676686c-gp2wq\" (UID: \"076e3a37-3f46-4197-9447-cf2b1ef9414d\") " pod="openshift-ingress/e2e-shared-a-8bae7d-openshift-default-7bf676686c-gp2wq"
Jul 16 11:36:24.055696 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:24.055406    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/076e3a37-3f46-4197-9447-cf2b1ef9414d-istio-podinfo\") pod \"e2e-shared-a-8bae7d-openshift-default-7bf676686c-gp2wq\" (UID: \"076e3a37-3f46-4197-9447-cf2b1ef9414d\") " pod="openshift-ingress/e2e-shared-a-8bae7d-openshift-default-7bf676686c-gp2wq"
Jul 16 11:36:24.055696 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:24.055440    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/076e3a37-3f46-4197-9447-cf2b1ef9414d-workload-socket\") pod \"e2e-shared-a-8bae7d-openshift-default-7bf676686c-gp2wq\" (UID: \"076e3a37-3f46-4197-9447-cf2b1ef9414d\") " pod="openshift-ingress/e2e-shared-a-8bae7d-openshift-default-7bf676686c-gp2wq"
Jul 16 11:36:24.055696 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:24.055464    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wltrf\" (UniqueName: \"kubernetes.io/projected/076e3a37-3f46-4197-9447-cf2b1ef9414d-kube-api-access-wltrf\") pod \"e2e-shared-a-8bae7d-openshift-default-7bf676686c-gp2wq\" (UID: \"076e3a37-3f46-4197-9447-cf2b1ef9414d\") " pod="openshift-ingress/e2e-shared-a-8bae7d-openshift-default-7bf676686c-gp2wq"
Jul 16 11:36:24.055696 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:24.055522    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/076e3a37-3f46-4197-9447-cf2b1ef9414d-credential-socket\") pod \"e2e-shared-a-8bae7d-openshift-default-7bf676686c-gp2wq\" (UID: \"076e3a37-3f46-4197-9447-cf2b1ef9414d\") " pod="openshift-ingress/e2e-shared-a-8bae7d-openshift-default-7bf676686c-gp2wq"
Jul 16 11:36:24.055900 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:24.055838    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/076e3a37-3f46-4197-9447-cf2b1ef9414d-istio-data\") pod \"e2e-shared-a-8bae7d-openshift-default-7bf676686c-gp2wq\" (UID: \"076e3a37-3f46-4197-9447-cf2b1ef9414d\") " pod="openshift-ingress/e2e-shared-a-8bae7d-openshift-default-7bf676686c-gp2wq"
Jul 16 11:36:24.055953 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:24.055897    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/076e3a37-3f46-4197-9447-cf2b1ef9414d-workload-socket\") pod \"e2e-shared-a-8bae7d-openshift-default-7bf676686c-gp2wq\" (UID: \"076e3a37-3f46-4197-9447-cf2b1ef9414d\") " pod="openshift-ingress/e2e-shared-a-8bae7d-openshift-default-7bf676686c-gp2wq"
Jul 16 11:36:24.055953 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:24.055915    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/076e3a37-3f46-4197-9447-cf2b1ef9414d-credential-socket\") pod \"e2e-shared-a-8bae7d-openshift-default-7bf676686c-gp2wq\" (UID: \"076e3a37-3f46-4197-9447-cf2b1ef9414d\") " pod="openshift-ingress/e2e-shared-a-8bae7d-openshift-default-7bf676686c-gp2wq"
Jul 16 11:36:24.055953 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:24.055943    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/076e3a37-3f46-4197-9447-cf2b1ef9414d-workload-certs\") pod \"e2e-shared-a-8bae7d-openshift-default-7bf676686c-gp2wq\" (UID: \"076e3a37-3f46-4197-9447-cf2b1ef9414d\") " pod="openshift-ingress/e2e-shared-a-8bae7d-openshift-default-7bf676686c-gp2wq"
Jul 16 11:36:24.056107 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:24.055986    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/076e3a37-3f46-4197-9447-cf2b1ef9414d-istio-token\") pod \"e2e-shared-a-8bae7d-openshift-default-7bf676686c-gp2wq\" (UID: \"076e3a37-3f46-4197-9447-cf2b1ef9414d\") " pod="openshift-ingress/e2e-shared-a-8bae7d-openshift-default-7bf676686c-gp2wq"
Jul 16 11:36:24.056272 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:24.056250    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/076e3a37-3f46-4197-9447-cf2b1ef9414d-workload-certs\") pod \"e2e-shared-a-8bae7d-openshift-default-7bf676686c-gp2wq\" (UID: \"076e3a37-3f46-4197-9447-cf2b1ef9414d\") " pod="openshift-ingress/e2e-shared-a-8bae7d-openshift-default-7bf676686c-gp2wq"
Jul 16 11:36:24.056470 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:24.056449    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/076e3a37-3f46-4197-9447-cf2b1ef9414d-istiod-ca-cert\") pod \"e2e-shared-a-8bae7d-openshift-default-7bf676686c-gp2wq\" (UID: \"076e3a37-3f46-4197-9447-cf2b1ef9414d\") " pod="openshift-ingress/e2e-shared-a-8bae7d-openshift-default-7bf676686c-gp2wq"
Jul 16 11:36:24.057776 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:24.057745    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/076e3a37-3f46-4197-9447-cf2b1ef9414d-istio-envoy\") pod \"e2e-shared-a-8bae7d-openshift-default-7bf676686c-gp2wq\" (UID: \"076e3a37-3f46-4197-9447-cf2b1ef9414d\") " pod="openshift-ingress/e2e-shared-a-8bae7d-openshift-default-7bf676686c-gp2wq"
Jul 16 11:36:24.058026 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:24.058007    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/076e3a37-3f46-4197-9447-cf2b1ef9414d-istio-podinfo\") pod \"e2e-shared-a-8bae7d-openshift-default-7bf676686c-gp2wq\" (UID: \"076e3a37-3f46-4197-9447-cf2b1ef9414d\") " pod="openshift-ingress/e2e-shared-a-8bae7d-openshift-default-7bf676686c-gp2wq"
Jul 16 11:36:24.064137 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:24.064105    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/076e3a37-3f46-4197-9447-cf2b1ef9414d-istio-token\") pod \"e2e-shared-a-8bae7d-openshift-default-7bf676686c-gp2wq\" (UID: \"076e3a37-3f46-4197-9447-cf2b1ef9414d\") " pod="openshift-ingress/e2e-shared-a-8bae7d-openshift-default-7bf676686c-gp2wq"
Jul 16 11:36:24.064247 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:24.064142    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wltrf\" (UniqueName: \"kubernetes.io/projected/076e3a37-3f46-4197-9447-cf2b1ef9414d-kube-api-access-wltrf\") pod \"e2e-shared-a-8bae7d-openshift-default-7bf676686c-gp2wq\" (UID: \"076e3a37-3f46-4197-9447-cf2b1ef9414d\") " pod="openshift-ingress/e2e-shared-a-8bae7d-openshift-default-7bf676686c-gp2wq"
Jul 16 11:36:24.102285 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:24.102247    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-8bae7d-openshift-default-7bf676686c-gp2wq"
Jul 16 11:36:24.244140 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:24.244093    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-8bae7d-openshift-default-7bf676686c-gp2wq"]
Jul 16 11:36:24.246904 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:36:24.246870    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod076e3a37_3f46_4197_9447_cf2b1ef9414d.slice/crio-6ee968da41fd2d847358fe306608d3f9c60c4fb24f4c3441c3997f5bff51feea WatchSource:0}: Error finding container 6ee968da41fd2d847358fe306608d3f9c60c4fb24f4c3441c3997f5bff51feea: Status 404 returned error can't find the container with id 6ee968da41fd2d847358fe306608d3f9c60c4fb24f4c3441c3997f5bff51feea
Jul 16 11:36:24.249077 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:24.249043    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 11:36:24.249191 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:24.249111    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 11:36:24.249191 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:24.249137    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 11:36:24.896039 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:24.896001    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-8bae7d-openshift-default-7bf676686c-gp2wq" event={"ID":"076e3a37-3f46-4197-9447-cf2b1ef9414d","Type":"ContainerStarted","Data":"27760542676d0006c0317df76af218a8d25da8325e3f1baf2d0e77d76b50cdd8"}
Jul 16 11:36:24.896439 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:24.896060    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-8bae7d-openshift-default-7bf676686c-gp2wq" event={"ID":"076e3a37-3f46-4197-9447-cf2b1ef9414d","Type":"ContainerStarted","Data":"6ee968da41fd2d847358fe306608d3f9c60c4fb24f4c3441c3997f5bff51feea"}
Jul 16 11:36:24.917940 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:24.917881    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-8bae7d-openshift-default-7bf676686c-gp2wq" podStartSLOduration=1.917861277 podStartE2EDuration="1.917861277s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 11:36:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 11:36:24.917405392 +0000 UTC m=+1977.551626846" watchObservedRunningTime="2026-07-16 11:36:24.917861277 +0000 UTC m=+1977.552082733"
Jul 16 11:36:25.103117 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:25.103067    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-8bae7d-openshift-default-7bf676686c-gp2wq"
Jul 16 11:36:25.108203 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:25.108175    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-8bae7d-openshift-default-7bf676686c-gp2wq"
Jul 16 11:36:25.899489 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:25.899458    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-8bae7d-bbcdb65dd-wrr89"]
Jul 16 11:36:25.902236 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:25.902215    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8bae7d-bbcdb65dd-wrr89"
Jul 16 11:36:25.903325 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:25.903300    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-8bae7d-openshift-default-7bf676686c-gp2wq"
Jul 16 11:36:25.904278 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:25.904259    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-8bae7d-openshift-default-7bf676686c-gp2wq"
Jul 16 11:36:25.904520 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:25.904507    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-8bae7d-serving-cert\""
Jul 16 11:36:25.914096 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:25.914068    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-8bae7d-bbcdb65dd-wrr89"]
Jul 16 11:36:25.976421 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:25.976378    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cbea2e56-7102-4fe4-8817-bef4b535edee-openshift-service-ca\") pod \"maas-api-e2e-shared-a-8bae7d-bbcdb65dd-wrr89\" (UID: \"cbea2e56-7102-4fe4-8817-bef4b535edee\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8bae7d-bbcdb65dd-wrr89"
Jul 16 11:36:25.976631 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:25.976576    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v2d2v\" (UniqueName: \"kubernetes.io/projected/cbea2e56-7102-4fe4-8817-bef4b535edee-kube-api-access-v2d2v\") pod \"maas-api-e2e-shared-a-8bae7d-bbcdb65dd-wrr89\" (UID: \"cbea2e56-7102-4fe4-8817-bef4b535edee\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8bae7d-bbcdb65dd-wrr89"
Jul 16 11:36:25.976711 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:25.976643    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cbea2e56-7102-4fe4-8817-bef4b535edee-maas-api-tls\") pod \"maas-api-e2e-shared-a-8bae7d-bbcdb65dd-wrr89\" (UID: \"cbea2e56-7102-4fe4-8817-bef4b535edee\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8bae7d-bbcdb65dd-wrr89"
Jul 16 11:36:26.077175 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:26.077121    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v2d2v\" (UniqueName: \"kubernetes.io/projected/cbea2e56-7102-4fe4-8817-bef4b535edee-kube-api-access-v2d2v\") pod \"maas-api-e2e-shared-a-8bae7d-bbcdb65dd-wrr89\" (UID: \"cbea2e56-7102-4fe4-8817-bef4b535edee\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8bae7d-bbcdb65dd-wrr89"
Jul 16 11:36:26.077378 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:26.077188    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cbea2e56-7102-4fe4-8817-bef4b535edee-maas-api-tls\") pod \"maas-api-e2e-shared-a-8bae7d-bbcdb65dd-wrr89\" (UID: \"cbea2e56-7102-4fe4-8817-bef4b535edee\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8bae7d-bbcdb65dd-wrr89"
Jul 16 11:36:26.077378 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:26.077278    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cbea2e56-7102-4fe4-8817-bef4b535edee-openshift-service-ca\") pod \"maas-api-e2e-shared-a-8bae7d-bbcdb65dd-wrr89\" (UID: \"cbea2e56-7102-4fe4-8817-bef4b535edee\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8bae7d-bbcdb65dd-wrr89"
Jul 16 11:36:26.077378 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:36:26.077347    2583 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-8bae7d-serving-cert: secret "maas-api-e2e-shared-a-8bae7d-serving-cert" not found
Jul 16 11:36:26.077548 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:36:26.077431    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cbea2e56-7102-4fe4-8817-bef4b535edee-maas-api-tls podName:cbea2e56-7102-4fe4-8817-bef4b535edee nodeName:}" failed. No retries permitted until 2026-07-16 11:36:26.577408265 +0000 UTC m=+1979.211629698 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/cbea2e56-7102-4fe4-8817-bef4b535edee-maas-api-tls") pod "maas-api-e2e-shared-a-8bae7d-bbcdb65dd-wrr89" (UID: "cbea2e56-7102-4fe4-8817-bef4b535edee") : secret "maas-api-e2e-shared-a-8bae7d-serving-cert" not found
Jul 16 11:36:26.078284 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:26.078165    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cbea2e56-7102-4fe4-8817-bef4b535edee-openshift-service-ca\") pod \"maas-api-e2e-shared-a-8bae7d-bbcdb65dd-wrr89\" (UID: \"cbea2e56-7102-4fe4-8817-bef4b535edee\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8bae7d-bbcdb65dd-wrr89"
Jul 16 11:36:26.089978 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:26.089942    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v2d2v\" (UniqueName: \"kubernetes.io/projected/cbea2e56-7102-4fe4-8817-bef4b535edee-kube-api-access-v2d2v\") pod \"maas-api-e2e-shared-a-8bae7d-bbcdb65dd-wrr89\" (UID: \"cbea2e56-7102-4fe4-8817-bef4b535edee\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8bae7d-bbcdb65dd-wrr89"
Jul 16 11:36:26.581716 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:26.581678    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cbea2e56-7102-4fe4-8817-bef4b535edee-maas-api-tls\") pod \"maas-api-e2e-shared-a-8bae7d-bbcdb65dd-wrr89\" (UID: \"cbea2e56-7102-4fe4-8817-bef4b535edee\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8bae7d-bbcdb65dd-wrr89"
Jul 16 11:36:26.584171 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:26.584150    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cbea2e56-7102-4fe4-8817-bef4b535edee-maas-api-tls\") pod \"maas-api-e2e-shared-a-8bae7d-bbcdb65dd-wrr89\" (UID: \"cbea2e56-7102-4fe4-8817-bef4b535edee\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8bae7d-bbcdb65dd-wrr89"
Jul 16 11:36:26.815198 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:26.815163    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8bae7d-bbcdb65dd-wrr89"
Jul 16 11:36:26.954501 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:26.954472    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-8bae7d-bbcdb65dd-wrr89"]
Jul 16 11:36:26.956774 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:36:26.956719    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcbea2e56_7102_4fe4_8817_bef4b535edee.slice/crio-6e0e8d369ec9dbab41f77af57a8ff0e02b103b7ea289ef300adc9c61e98c37fb WatchSource:0}: Error finding container 6e0e8d369ec9dbab41f77af57a8ff0e02b103b7ea289ef300adc9c61e98c37fb: Status 404 returned error can't find the container with id 6e0e8d369ec9dbab41f77af57a8ff0e02b103b7ea289ef300adc9c61e98c37fb
Jul 16 11:36:27.914844 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:27.914805    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8bae7d-bbcdb65dd-wrr89" event={"ID":"cbea2e56-7102-4fe4-8817-bef4b535edee","Type":"ContainerStarted","Data":"1370634a9b74c20b21c145e6fb01236a95e2420254a75b508caf3508d57064b6"}
Jul 16 11:36:27.915033 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:27.914911    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8bae7d-bbcdb65dd-wrr89" event={"ID":"cbea2e56-7102-4fe4-8817-bef4b535edee","Type":"ContainerStarted","Data":"6e0e8d369ec9dbab41f77af57a8ff0e02b103b7ea289ef300adc9c61e98c37fb"}
Jul 16 11:36:27.915085 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:27.915066    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8bae7d-bbcdb65dd-wrr89"
Jul 16 11:36:27.934465 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:27.934407    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8bae7d-bbcdb65dd-wrr89" podStartSLOduration=2.572436819 podStartE2EDuration="2.934391952s" totalImagesPullingTime="361.955133ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 11:36:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 11:36:27.932934528 +0000 UTC m=+1980.567156007" watchObservedRunningTime="2026-07-16 11:36:27.934391952 +0000 UTC m=+1980.568613405"
Jul 16 11:36:33.563290 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:33.563251    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-a47fa3-7b769fd796-mptbl"]
Jul 16 11:36:33.566200 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:33.566177    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-a47fa3-7b769fd796-mptbl"
Jul 16 11:36:33.568640 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:33.568617    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-a47fa3-serving-cert\""
Jul 16 11:36:33.581762 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:33.581730    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-a47fa3-7b769fd796-mptbl"]
Jul 16 11:36:33.636360 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:33.636320    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4c3bdeb8-efaa-407a-9bea-5f639e284f34-openshift-service-ca\") pod \"maas-api-e2e-shared-b-a47fa3-7b769fd796-mptbl\" (UID: \"4c3bdeb8-efaa-407a-9bea-5f639e284f34\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-a47fa3-7b769fd796-mptbl"
Jul 16 11:36:33.636783 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:33.636752    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4c3bdeb8-efaa-407a-9bea-5f639e284f34-maas-api-tls\") pod \"maas-api-e2e-shared-b-a47fa3-7b769fd796-mptbl\" (UID: \"4c3bdeb8-efaa-407a-9bea-5f639e284f34\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-a47fa3-7b769fd796-mptbl"
Jul 16 11:36:33.637027 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:33.637009    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n9zcz\" (UniqueName: \"kubernetes.io/projected/4c3bdeb8-efaa-407a-9bea-5f639e284f34-kube-api-access-n9zcz\") pod \"maas-api-e2e-shared-b-a47fa3-7b769fd796-mptbl\" (UID: \"4c3bdeb8-efaa-407a-9bea-5f639e284f34\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-a47fa3-7b769fd796-mptbl"
Jul 16 11:36:33.738620 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:33.738091    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4c3bdeb8-efaa-407a-9bea-5f639e284f34-openshift-service-ca\") pod \"maas-api-e2e-shared-b-a47fa3-7b769fd796-mptbl\" (UID: \"4c3bdeb8-efaa-407a-9bea-5f639e284f34\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-a47fa3-7b769fd796-mptbl"
Jul 16 11:36:33.738620 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:33.738166    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4c3bdeb8-efaa-407a-9bea-5f639e284f34-maas-api-tls\") pod \"maas-api-e2e-shared-b-a47fa3-7b769fd796-mptbl\" (UID: \"4c3bdeb8-efaa-407a-9bea-5f639e284f34\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-a47fa3-7b769fd796-mptbl"
Jul 16 11:36:33.738620 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:33.738207    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n9zcz\" (UniqueName: \"kubernetes.io/projected/4c3bdeb8-efaa-407a-9bea-5f639e284f34-kube-api-access-n9zcz\") pod \"maas-api-e2e-shared-b-a47fa3-7b769fd796-mptbl\" (UID: \"4c3bdeb8-efaa-407a-9bea-5f639e284f34\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-a47fa3-7b769fd796-mptbl"
Jul 16 11:36:33.739871 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:33.739404    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4c3bdeb8-efaa-407a-9bea-5f639e284f34-openshift-service-ca\") pod \"maas-api-e2e-shared-b-a47fa3-7b769fd796-mptbl\" (UID: \"4c3bdeb8-efaa-407a-9bea-5f639e284f34\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-a47fa3-7b769fd796-mptbl"
Jul 16 11:36:33.739871 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:36:33.739517    2583 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-a47fa3-serving-cert: secret "maas-api-e2e-shared-b-a47fa3-serving-cert" not found
Jul 16 11:36:33.739871 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:36:33.739642    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4c3bdeb8-efaa-407a-9bea-5f639e284f34-maas-api-tls podName:4c3bdeb8-efaa-407a-9bea-5f639e284f34 nodeName:}" failed. No retries permitted until 2026-07-16 11:36:34.23962267 +0000 UTC m=+1986.873844108 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/4c3bdeb8-efaa-407a-9bea-5f639e284f34-maas-api-tls") pod "maas-api-e2e-shared-b-a47fa3-7b769fd796-mptbl" (UID: "4c3bdeb8-efaa-407a-9bea-5f639e284f34") : secret "maas-api-e2e-shared-b-a47fa3-serving-cert" not found
Jul 16 11:36:33.752768 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:33.750234    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n9zcz\" (UniqueName: \"kubernetes.io/projected/4c3bdeb8-efaa-407a-9bea-5f639e284f34-kube-api-access-n9zcz\") pod \"maas-api-e2e-shared-b-a47fa3-7b769fd796-mptbl\" (UID: \"4c3bdeb8-efaa-407a-9bea-5f639e284f34\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-a47fa3-7b769fd796-mptbl"
Jul 16 11:36:33.935591 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:33.935549    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8bae7d-bbcdb65dd-wrr89"
Jul 16 11:36:34.246583 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:34.246484    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4c3bdeb8-efaa-407a-9bea-5f639e284f34-maas-api-tls\") pod \"maas-api-e2e-shared-b-a47fa3-7b769fd796-mptbl\" (UID: \"4c3bdeb8-efaa-407a-9bea-5f639e284f34\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-a47fa3-7b769fd796-mptbl"
Jul 16 11:36:34.248982 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:34.248953    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4c3bdeb8-efaa-407a-9bea-5f639e284f34-maas-api-tls\") pod \"maas-api-e2e-shared-b-a47fa3-7b769fd796-mptbl\" (UID: \"4c3bdeb8-efaa-407a-9bea-5f639e284f34\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-a47fa3-7b769fd796-mptbl"
Jul 16 11:36:34.479286 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:34.479247    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-a47fa3-7b769fd796-mptbl"
Jul 16 11:36:34.671853 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:34.671818    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-a47fa3-7b769fd796-mptbl"]
Jul 16 11:36:34.675303 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:36:34.675276    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c3bdeb8_efaa_407a_9bea_5f639e284f34.slice/crio-f757eec4618c86ee033dd5013d96707de098eb014b2373b3e440c4bb067c68e0 WatchSource:0}: Error finding container f757eec4618c86ee033dd5013d96707de098eb014b2373b3e440c4bb067c68e0: Status 404 returned error can't find the container with id f757eec4618c86ee033dd5013d96707de098eb014b2373b3e440c4bb067c68e0
Jul 16 11:36:34.955572 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:34.955537    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-a47fa3-7b769fd796-mptbl" event={"ID":"4c3bdeb8-efaa-407a-9bea-5f639e284f34","Type":"ContainerStarted","Data":"f757eec4618c86ee033dd5013d96707de098eb014b2373b3e440c4bb067c68e0"}
Jul 16 11:36:35.962061 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:35.962023    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-a47fa3-7b769fd796-mptbl" event={"ID":"4c3bdeb8-efaa-407a-9bea-5f639e284f34","Type":"ContainerStarted","Data":"3e422bb99617b19c6110f65675c1b3ecdf3052924c76e37e1447eb563335174e"}
Jul 16 11:36:35.962537 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:35.962102    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-a47fa3-7b769fd796-mptbl"
Jul 16 11:36:35.980480 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:35.980430    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-a47fa3-7b769fd796-mptbl" podStartSLOduration=2.627301412 podStartE2EDuration="2.980413713s" totalImagesPullingTime="353.112301ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 11:36:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 11:36:35.979099861 +0000 UTC m=+1988.613321315" watchObservedRunningTime="2026-07-16 11:36:35.980413713 +0000 UTC m=+1988.614635186"
Jul 16 11:36:40.904930 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:40.904895    2583 patch_prober.go:28] interesting pod/e2e-shared-a-8bae7d-openshift-default-7bf676686c-gp2wq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body=
Jul 16 11:36:40.905388 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:40.904956    2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-8bae7d-openshift-default-7bf676686c-gp2wq" podUID="076e3a37-3f46-4197-9447-cf2b1ef9414d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused"
Jul 16 11:36:40.994776 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:40.994742    2583 generic.go:356] "Generic (PLEG): container finished" podID="076e3a37-3f46-4197-9447-cf2b1ef9414d" containerID="27760542676d0006c0317df76af218a8d25da8325e3f1baf2d0e77d76b50cdd8" exitCode=0
Jul 16 11:36:40.994948 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:40.994801    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-8bae7d-openshift-default-7bf676686c-gp2wq" event={"ID":"076e3a37-3f46-4197-9447-cf2b1ef9414d","Type":"ContainerDied","Data":"27760542676d0006c0317df76af218a8d25da8325e3f1baf2d0e77d76b50cdd8"}
Jul 16 11:36:40.995227 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:40.995212    2583 scope.go:117] "RemoveContainer" containerID="27760542676d0006c0317df76af218a8d25da8325e3f1baf2d0e77d76b50cdd8"
Jul 16 11:36:40.996247 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:40.996222    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 11:36:40.996313 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:40.996288    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 11:36:40.996349 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:40.996318    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 11:36:41.980312 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:41.980281    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-a47fa3-7b769fd796-mptbl"
Jul 16 11:36:42.001603 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:42.001561    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-8bae7d-openshift-default-7bf676686c-gp2wq" event={"ID":"076e3a37-3f46-4197-9447-cf2b1ef9414d","Type":"ContainerStarted","Data":"510d46bf4051b4d503c124082cc7551542ad01495ac930f474aab30220bf7ba8"}
Jul 16 11:36:42.102991 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:42.102947    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-8bae7d-openshift-default-7bf676686c-gp2wq"
Jul 16 11:36:42.103229 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:42.103206    2583 patch_prober.go:28] interesting pod/e2e-shared-a-8bae7d-openshift-default-7bf676686c-gp2wq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body=
Jul 16 11:36:42.103308 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:42.103274    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-8bae7d-openshift-default-7bf676686c-gp2wq" podUID="076e3a37-3f46-4197-9447-cf2b1ef9414d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused"
Jul 16 11:36:43.103108 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:43.103069    2583 patch_prober.go:28] interesting pod/e2e-shared-a-8bae7d-openshift-default-7bf676686c-gp2wq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body=
Jul 16 11:36:43.103542 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:43.103138    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-8bae7d-openshift-default-7bf676686c-gp2wq" podUID="076e3a37-3f46-4197-9447-cf2b1ef9414d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused"
Jul 16 11:36:44.103701 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:44.103656    2583 patch_prober.go:28] interesting pod/e2e-shared-a-8bae7d-openshift-default-7bf676686c-gp2wq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body=
Jul 16 11:36:44.104118 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:44.103757    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-8bae7d-openshift-default-7bf676686c-gp2wq" podUID="076e3a37-3f46-4197-9447-cf2b1ef9414d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused"
Jul 16 11:36:45.103074 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:45.103032    2583 patch_prober.go:28] interesting pod/e2e-shared-a-8bae7d-openshift-default-7bf676686c-gp2wq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body=
Jul 16 11:36:45.103272 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:45.103104    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-8bae7d-openshift-default-7bf676686c-gp2wq" podUID="076e3a37-3f46-4197-9447-cf2b1ef9414d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused"
Jul 16 11:36:46.107424 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:46.107391    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-8bae7d-openshift-default-7bf676686c-gp2wq"
Jul 16 11:36:46.107953 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:46.107708    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-8bae7d-openshift-default-7bf676686c-gp2wq"
Jul 16 11:36:46.108404 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:36:46.108387    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-8bae7d-openshift-default-7bf676686c-gp2wq"
Jul 16 11:38:29.285159 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:38:29.285125    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-9lm5s_f64f15a3-5a9f-4737-bc4f-73d5ec311e41/console-operator/1.log"
Jul 16 11:38:29.290568 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:38:29.290546    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-9lm5s_f64f15a3-5a9f-4737-bc4f-73d5ec311e41/console-operator/1.log"
Jul 16 11:38:29.291800 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:38:29.291778    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2db6t_69aed169-e8af-429d-99c3-3b20be64e351/ovn-acl-logging/0.log"
Jul 16 11:38:29.296586 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:38:29.296569    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2db6t_69aed169-e8af-429d-99c3-3b20be64e351/ovn-acl-logging/0.log"
Jul 16 11:42:30.263079 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:42:30.262882    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-8bae7d-bbcdb65dd-wrr89"]
Jul 16 11:42:30.263963 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:42:30.263510    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8bae7d-bbcdb65dd-wrr89" podUID="cbea2e56-7102-4fe4-8817-bef4b535edee" containerName="maas-api" containerID="cri-o://1370634a9b74c20b21c145e6fb01236a95e2420254a75b508caf3508d57064b6" gracePeriod=30
Jul 16 11:42:30.710844 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:42:30.710807    2583 generic.go:356] "Generic (PLEG): container finished" podID="cbea2e56-7102-4fe4-8817-bef4b535edee" containerID="1370634a9b74c20b21c145e6fb01236a95e2420254a75b508caf3508d57064b6" exitCode=0
Jul 16 11:42:30.710985 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:42:30.710876    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8bae7d-bbcdb65dd-wrr89" event={"ID":"cbea2e56-7102-4fe4-8817-bef4b535edee","Type":"ContainerDied","Data":"1370634a9b74c20b21c145e6fb01236a95e2420254a75b508caf3508d57064b6"}
Jul 16 11:42:30.710985 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:42:30.710931    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8bae7d-bbcdb65dd-wrr89" event={"ID":"cbea2e56-7102-4fe4-8817-bef4b535edee","Type":"ContainerDied","Data":"6e0e8d369ec9dbab41f77af57a8ff0e02b103b7ea289ef300adc9c61e98c37fb"}
Jul 16 11:42:30.710985 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:42:30.710949    2583 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6e0e8d369ec9dbab41f77af57a8ff0e02b103b7ea289ef300adc9c61e98c37fb"
Jul 16 11:42:30.724222 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:42:30.724198    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8bae7d-bbcdb65dd-wrr89"
Jul 16 11:42:30.835943 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:42:30.835843    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cbea2e56-7102-4fe4-8817-bef4b535edee-maas-api-tls\") pod \"cbea2e56-7102-4fe4-8817-bef4b535edee\" (UID: \"cbea2e56-7102-4fe4-8817-bef4b535edee\") "
Jul 16 11:42:30.835943 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:42:30.835903    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v2d2v\" (UniqueName: \"kubernetes.io/projected/cbea2e56-7102-4fe4-8817-bef4b535edee-kube-api-access-v2d2v\") pod \"cbea2e56-7102-4fe4-8817-bef4b535edee\" (UID: \"cbea2e56-7102-4fe4-8817-bef4b535edee\") "
Jul 16 11:42:30.836194 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:42:30.835975    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cbea2e56-7102-4fe4-8817-bef4b535edee-openshift-service-ca\") pod \"cbea2e56-7102-4fe4-8817-bef4b535edee\" (UID: \"cbea2e56-7102-4fe4-8817-bef4b535edee\") "
Jul 16 11:42:30.836373 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:42:30.836340    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cbea2e56-7102-4fe4-8817-bef4b535edee-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "cbea2e56-7102-4fe4-8817-bef4b535edee" (UID: "cbea2e56-7102-4fe4-8817-bef4b535edee"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 11:42:30.837994 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:42:30.837972    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cbea2e56-7102-4fe4-8817-bef4b535edee-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "cbea2e56-7102-4fe4-8817-bef4b535edee" (UID: "cbea2e56-7102-4fe4-8817-bef4b535edee"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 11:42:30.838071 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:42:30.838009    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cbea2e56-7102-4fe4-8817-bef4b535edee-kube-api-access-v2d2v" (OuterVolumeSpecName: "kube-api-access-v2d2v") pod "cbea2e56-7102-4fe4-8817-bef4b535edee" (UID: "cbea2e56-7102-4fe4-8817-bef4b535edee"). InnerVolumeSpecName "kube-api-access-v2d2v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 11:42:30.937311 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:42:30.937274    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cbea2e56-7102-4fe4-8817-bef4b535edee-maas-api-tls\") on node \"ip-10-0-129-28.ec2.internal\" DevicePath \"\""
Jul 16 11:42:30.937311 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:42:30.937306    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v2d2v\" (UniqueName: \"kubernetes.io/projected/cbea2e56-7102-4fe4-8817-bef4b535edee-kube-api-access-v2d2v\") on node \"ip-10-0-129-28.ec2.internal\" DevicePath \"\""
Jul 16 11:42:30.937311 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:42:30.937316    2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cbea2e56-7102-4fe4-8817-bef4b535edee-openshift-service-ca\") on node \"ip-10-0-129-28.ec2.internal\" DevicePath \"\""
Jul 16 11:42:31.715616 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:42:31.715563    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-8bae7d-bbcdb65dd-wrr89"
Jul 16 11:42:31.737374 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:42:31.737334    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-8bae7d-bbcdb65dd-wrr89"]
Jul 16 11:42:31.740814 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:42:31.740781    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-8bae7d-bbcdb65dd-wrr89"]
Jul 16 11:42:32.039215 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:42:32.039120    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-8bae7d-openshift-default-7bf676686c-gp2wq"]
Jul 16 11:42:32.039388 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:42:32.039330    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-8bae7d-openshift-default-7bf676686c-gp2wq" podUID="076e3a37-3f46-4197-9447-cf2b1ef9414d" containerName="istio-proxy" containerID="cri-o://510d46bf4051b4d503c124082cc7551542ad01495ac930f474aab30220bf7ba8" gracePeriod=30
Jul 16 11:42:33.229003 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:42:33.228968    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cbea2e56-7102-4fe4-8817-bef4b535edee" path="/var/lib/kubelet/pods/cbea2e56-7102-4fe4-8817-bef4b535edee/volumes"
Jul 16 11:42:37.293479 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:42:37.293445    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-8bae7d-openshift-default-7bf676686c-gp2wq"
Jul 16 11:42:37.394532 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:42:37.394501    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wltrf\" (UniqueName: \"kubernetes.io/projected/076e3a37-3f46-4197-9447-cf2b1ef9414d-kube-api-access-wltrf\") pod \"076e3a37-3f46-4197-9447-cf2b1ef9414d\" (UID: \"076e3a37-3f46-4197-9447-cf2b1ef9414d\") "
Jul 16 11:42:37.394720 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:42:37.394549    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/076e3a37-3f46-4197-9447-cf2b1ef9414d-istio-podinfo\") pod \"076e3a37-3f46-4197-9447-cf2b1ef9414d\" (UID: \"076e3a37-3f46-4197-9447-cf2b1ef9414d\") "
Jul 16 11:42:37.394720 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:42:37.394584    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/076e3a37-3f46-4197-9447-cf2b1ef9414d-workload-certs\") pod \"076e3a37-3f46-4197-9447-cf2b1ef9414d\" (UID: \"076e3a37-3f46-4197-9447-cf2b1ef9414d\") "
Jul 16 11:42:37.394720 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:42:37.394678    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/076e3a37-3f46-4197-9447-cf2b1ef9414d-credential-socket\") pod \"076e3a37-3f46-4197-9447-cf2b1ef9414d\" (UID: \"076e3a37-3f46-4197-9447-cf2b1ef9414d\") "
Jul 16 11:42:37.394888 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:42:37.394723    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/076e3a37-3f46-4197-9447-cf2b1ef9414d-istio-data\") pod \"076e3a37-3f46-4197-9447-cf2b1ef9414d\" (UID: \"076e3a37-3f46-4197-9447-cf2b1ef9414d\") "
Jul 16 11:42:37.394888 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:42:37.394749    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/076e3a37-3f46-4197-9447-cf2b1ef9414d-workload-socket\") pod \"076e3a37-3f46-4197-9447-cf2b1ef9414d\" (UID: \"076e3a37-3f46-4197-9447-cf2b1ef9414d\") "
Jul 16 11:42:37.394888 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:42:37.394799    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/076e3a37-3f46-4197-9447-cf2b1ef9414d-istiod-ca-cert\") pod \"076e3a37-3f46-4197-9447-cf2b1ef9414d\" (UID: \"076e3a37-3f46-4197-9447-cf2b1ef9414d\") "
Jul 16 11:42:37.394888 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:42:37.394829    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/076e3a37-3f46-4197-9447-cf2b1ef9414d-istio-envoy\") pod \"076e3a37-3f46-4197-9447-cf2b1ef9414d\" (UID: \"076e3a37-3f46-4197-9447-cf2b1ef9414d\") "
Jul 16 11:42:37.394888 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:42:37.394858    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/076e3a37-3f46-4197-9447-cf2b1ef9414d-istio-token\") pod \"076e3a37-3f46-4197-9447-cf2b1ef9414d\" (UID: \"076e3a37-3f46-4197-9447-cf2b1ef9414d\") "
Jul 16 11:42:37.395130 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:42:37.394996    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/076e3a37-3f46-4197-9447-cf2b1ef9414d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "076e3a37-3f46-4197-9447-cf2b1ef9414d" (UID: "076e3a37-3f46-4197-9447-cf2b1ef9414d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 11:42:37.395130 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:42:37.395010    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/076e3a37-3f46-4197-9447-cf2b1ef9414d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "076e3a37-3f46-4197-9447-cf2b1ef9414d" (UID: "076e3a37-3f46-4197-9447-cf2b1ef9414d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 11:42:37.395235 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:42:37.395172    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/076e3a37-3f46-4197-9447-cf2b1ef9414d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "076e3a37-3f46-4197-9447-cf2b1ef9414d" (UID: "076e3a37-3f46-4197-9447-cf2b1ef9414d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 11:42:37.395297 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:42:37.395283    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/076e3a37-3f46-4197-9447-cf2b1ef9414d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "076e3a37-3f46-4197-9447-cf2b1ef9414d" (UID: "076e3a37-3f46-4197-9447-cf2b1ef9414d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 11:42:37.395415 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:42:37.395392    2583 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/076e3a37-3f46-4197-9447-cf2b1ef9414d-istiod-ca-cert\") on node \"ip-10-0-129-28.ec2.internal\" DevicePath \"\""
Jul 16 11:42:37.395474 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:42:37.395421    2583 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/076e3a37-3f46-4197-9447-cf2b1ef9414d-workload-certs\") on node \"ip-10-0-129-28.ec2.internal\" DevicePath \"\""
Jul 16 11:42:37.395474 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:42:37.395437    2583 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/076e3a37-3f46-4197-9447-cf2b1ef9414d-credential-socket\") on node \"ip-10-0-129-28.ec2.internal\" DevicePath \"\""
Jul 16 11:42:37.395474 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:42:37.395450    2583 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/076e3a37-3f46-4197-9447-cf2b1ef9414d-workload-socket\") on node \"ip-10-0-129-28.ec2.internal\" DevicePath \"\""
Jul 16 11:42:37.395697 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:42:37.395668    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/076e3a37-3f46-4197-9447-cf2b1ef9414d-istio-data" (OuterVolumeSpecName: "istio-data") pod "076e3a37-3f46-4197-9447-cf2b1ef9414d" (UID: "076e3a37-3f46-4197-9447-cf2b1ef9414d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 11:42:37.397004 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:42:37.396973    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/076e3a37-3f46-4197-9447-cf2b1ef9414d-kube-api-access-wltrf" (OuterVolumeSpecName: "kube-api-access-wltrf") pod "076e3a37-3f46-4197-9447-cf2b1ef9414d" (UID: "076e3a37-3f46-4197-9447-cf2b1ef9414d"). InnerVolumeSpecName "kube-api-access-wltrf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 11:42:37.397307 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:42:37.397290    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/076e3a37-3f46-4197-9447-cf2b1ef9414d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "076e3a37-3f46-4197-9447-cf2b1ef9414d" (UID: "076e3a37-3f46-4197-9447-cf2b1ef9414d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 16 11:42:37.397368 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:42:37.397304    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/076e3a37-3f46-4197-9447-cf2b1ef9414d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "076e3a37-3f46-4197-9447-cf2b1ef9414d" (UID: "076e3a37-3f46-4197-9447-cf2b1ef9414d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 11:42:37.397476 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:42:37.397458    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/076e3a37-3f46-4197-9447-cf2b1ef9414d-istio-token" (OuterVolumeSpecName: "istio-token") pod "076e3a37-3f46-4197-9447-cf2b1ef9414d" (UID: "076e3a37-3f46-4197-9447-cf2b1ef9414d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 11:42:37.496326 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:42:37.496286    2583 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/076e3a37-3f46-4197-9447-cf2b1ef9414d-istio-data\") on node \"ip-10-0-129-28.ec2.internal\" DevicePath \"\""
Jul 16 11:42:37.496326 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:42:37.496319    2583 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/076e3a37-3f46-4197-9447-cf2b1ef9414d-istio-envoy\") on node \"ip-10-0-129-28.ec2.internal\" DevicePath \"\""
Jul 16 11:42:37.496326 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:42:37.496327    2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/076e3a37-3f46-4197-9447-cf2b1ef9414d-istio-token\") on node \"ip-10-0-129-28.ec2.internal\" DevicePath \"\""
Jul 16 11:42:37.496326 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:42:37.496336    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wltrf\" (UniqueName: \"kubernetes.io/projected/076e3a37-3f46-4197-9447-cf2b1ef9414d-kube-api-access-wltrf\") on node \"ip-10-0-129-28.ec2.internal\" DevicePath \"\""
Jul 16 11:42:37.496631 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:42:37.496345    2583 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/076e3a37-3f46-4197-9447-cf2b1ef9414d-istio-podinfo\") on node \"ip-10-0-129-28.ec2.internal\" DevicePath \"\""
Jul 16 11:42:37.745775 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:42:37.745687    2583 generic.go:356] "Generic (PLEG): container finished" podID="076e3a37-3f46-4197-9447-cf2b1ef9414d" containerID="510d46bf4051b4d503c124082cc7551542ad01495ac930f474aab30220bf7ba8" exitCode=0
Jul 16 11:42:37.745775 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:42:37.745738    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-8bae7d-openshift-default-7bf676686c-gp2wq"
Jul 16 11:42:37.745975 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:42:37.745767    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-8bae7d-openshift-default-7bf676686c-gp2wq" event={"ID":"076e3a37-3f46-4197-9447-cf2b1ef9414d","Type":"ContainerDied","Data":"510d46bf4051b4d503c124082cc7551542ad01495ac930f474aab30220bf7ba8"}
Jul 16 11:42:37.745975 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:42:37.745819    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-8bae7d-openshift-default-7bf676686c-gp2wq" event={"ID":"076e3a37-3f46-4197-9447-cf2b1ef9414d","Type":"ContainerDied","Data":"6ee968da41fd2d847358fe306608d3f9c60c4fb24f4c3441c3997f5bff51feea"}
Jul 16 11:42:37.745975 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:42:37.745842    2583 scope.go:117] "RemoveContainer" containerID="510d46bf4051b4d503c124082cc7551542ad01495ac930f474aab30220bf7ba8"
Jul 16 11:42:37.755130 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:42:37.755112    2583 scope.go:117] "RemoveContainer" containerID="27760542676d0006c0317df76af218a8d25da8325e3f1baf2d0e77d76b50cdd8"
Jul 16 11:42:37.764166 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:42:37.764145    2583 scope.go:117] "RemoveContainer" containerID="510d46bf4051b4d503c124082cc7551542ad01495ac930f474aab30220bf7ba8"
Jul 16 11:42:37.764453 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:42:37.764434    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"510d46bf4051b4d503c124082cc7551542ad01495ac930f474aab30220bf7ba8\": container with ID starting with 510d46bf4051b4d503c124082cc7551542ad01495ac930f474aab30220bf7ba8 not found: ID does not exist" containerID="510d46bf4051b4d503c124082cc7551542ad01495ac930f474aab30220bf7ba8"
Jul 16 11:42:37.764501 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:42:37.764464    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"510d46bf4051b4d503c124082cc7551542ad01495ac930f474aab30220bf7ba8"} err="failed to get container status \"510d46bf4051b4d503c124082cc7551542ad01495ac930f474aab30220bf7ba8\": rpc error: code = NotFound desc = could not find container \"510d46bf4051b4d503c124082cc7551542ad01495ac930f474aab30220bf7ba8\": container with ID starting with 510d46bf4051b4d503c124082cc7551542ad01495ac930f474aab30220bf7ba8 not found: ID does not exist"
Jul 16 11:42:37.764501 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:42:37.764481    2583 scope.go:117] "RemoveContainer" containerID="27760542676d0006c0317df76af218a8d25da8325e3f1baf2d0e77d76b50cdd8"
Jul 16 11:42:37.764731 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:42:37.764713    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"27760542676d0006c0317df76af218a8d25da8325e3f1baf2d0e77d76b50cdd8\": container with ID starting with 27760542676d0006c0317df76af218a8d25da8325e3f1baf2d0e77d76b50cdd8 not found: ID does not exist" containerID="27760542676d0006c0317df76af218a8d25da8325e3f1baf2d0e77d76b50cdd8"
Jul 16 11:42:37.764769 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:42:37.764735    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"27760542676d0006c0317df76af218a8d25da8325e3f1baf2d0e77d76b50cdd8"} err="failed to get container status \"27760542676d0006c0317df76af218a8d25da8325e3f1baf2d0e77d76b50cdd8\": rpc error: code = NotFound desc = could not find container \"27760542676d0006c0317df76af218a8d25da8325e3f1baf2d0e77d76b50cdd8\": container with ID starting with 27760542676d0006c0317df76af218a8d25da8325e3f1baf2d0e77d76b50cdd8 not found: ID does not exist"
Jul 16 11:42:37.774420 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:42:37.774383    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-8bae7d-openshift-default-7bf676686c-gp2wq"]
Jul 16 11:42:37.777310 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:42:37.777283    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-8bae7d-openshift-default-7bf676686c-gp2wq"]
Jul 16 11:42:38.427432 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:42:38.427399    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-a47fa3-7b769fd796-mptbl"]
Jul 16 11:42:38.427871 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:42:38.427666    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-a47fa3-7b769fd796-mptbl" podUID="4c3bdeb8-efaa-407a-9bea-5f639e284f34" containerName="maas-api" containerID="cri-o://3e422bb99617b19c6110f65675c1b3ecdf3052924c76e37e1447eb563335174e" gracePeriod=30
Jul 16 11:42:38.684588 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:42:38.684518    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-a47fa3-7b769fd796-mptbl"
Jul 16 11:42:38.751934 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:42:38.751895    2583 generic.go:356] "Generic (PLEG): container finished" podID="4c3bdeb8-efaa-407a-9bea-5f639e284f34" containerID="3e422bb99617b19c6110f65675c1b3ecdf3052924c76e37e1447eb563335174e" exitCode=0
Jul 16 11:42:38.752122 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:42:38.751939    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-a47fa3-7b769fd796-mptbl"
Jul 16 11:42:38.752122 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:42:38.751981    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-a47fa3-7b769fd796-mptbl" event={"ID":"4c3bdeb8-efaa-407a-9bea-5f639e284f34","Type":"ContainerDied","Data":"3e422bb99617b19c6110f65675c1b3ecdf3052924c76e37e1447eb563335174e"}
Jul 16 11:42:38.752122 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:42:38.752033    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-a47fa3-7b769fd796-mptbl" event={"ID":"4c3bdeb8-efaa-407a-9bea-5f639e284f34","Type":"ContainerDied","Data":"f757eec4618c86ee033dd5013d96707de098eb014b2373b3e440c4bb067c68e0"}
Jul 16 11:42:38.752122 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:42:38.752056    2583 scope.go:117] "RemoveContainer" containerID="3e422bb99617b19c6110f65675c1b3ecdf3052924c76e37e1447eb563335174e"
Jul 16 11:42:38.760930 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:42:38.760911    2583 scope.go:117] "RemoveContainer" containerID="3e422bb99617b19c6110f65675c1b3ecdf3052924c76e37e1447eb563335174e"
Jul 16 11:42:38.761212 ip-10-0-129-28 kubenswrapper[2583]: E0716 11:42:38.761189    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3e422bb99617b19c6110f65675c1b3ecdf3052924c76e37e1447eb563335174e\": container with ID starting with 3e422bb99617b19c6110f65675c1b3ecdf3052924c76e37e1447eb563335174e not found: ID does not exist" containerID="3e422bb99617b19c6110f65675c1b3ecdf3052924c76e37e1447eb563335174e"
Jul 16 11:42:38.761259 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:42:38.761221    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3e422bb99617b19c6110f65675c1b3ecdf3052924c76e37e1447eb563335174e"} err="failed to get container status \"3e422bb99617b19c6110f65675c1b3ecdf3052924c76e37e1447eb563335174e\": rpc error: code = NotFound desc = could not find container \"3e422bb99617b19c6110f65675c1b3ecdf3052924c76e37e1447eb563335174e\": container with ID starting with 3e422bb99617b19c6110f65675c1b3ecdf3052924c76e37e1447eb563335174e not found: ID does not exist"
Jul 16 11:42:38.808168 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:42:38.808131    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n9zcz\" (UniqueName: \"kubernetes.io/projected/4c3bdeb8-efaa-407a-9bea-5f639e284f34-kube-api-access-n9zcz\") pod \"4c3bdeb8-efaa-407a-9bea-5f639e284f34\" (UID: \"4c3bdeb8-efaa-407a-9bea-5f639e284f34\") "
Jul 16 11:42:38.808322 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:42:38.808198    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4c3bdeb8-efaa-407a-9bea-5f639e284f34-openshift-service-ca\") pod \"4c3bdeb8-efaa-407a-9bea-5f639e284f34\" (UID: \"4c3bdeb8-efaa-407a-9bea-5f639e284f34\") "
Jul 16 11:42:38.808322 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:42:38.808221    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4c3bdeb8-efaa-407a-9bea-5f639e284f34-maas-api-tls\") pod \"4c3bdeb8-efaa-407a-9bea-5f639e284f34\" (UID: \"4c3bdeb8-efaa-407a-9bea-5f639e284f34\") "
Jul 16 11:42:38.808563 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:42:38.808540    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4c3bdeb8-efaa-407a-9bea-5f639e284f34-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "4c3bdeb8-efaa-407a-9bea-5f639e284f34" (UID: "4c3bdeb8-efaa-407a-9bea-5f639e284f34"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 11:42:38.810269 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:42:38.810249    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4c3bdeb8-efaa-407a-9bea-5f639e284f34-kube-api-access-n9zcz" (OuterVolumeSpecName: "kube-api-access-n9zcz") pod "4c3bdeb8-efaa-407a-9bea-5f639e284f34" (UID: "4c3bdeb8-efaa-407a-9bea-5f639e284f34"). InnerVolumeSpecName "kube-api-access-n9zcz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 11:42:38.810374 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:42:38.810333    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4c3bdeb8-efaa-407a-9bea-5f639e284f34-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "4c3bdeb8-efaa-407a-9bea-5f639e284f34" (UID: "4c3bdeb8-efaa-407a-9bea-5f639e284f34"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 11:42:38.908944 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:42:38.908904    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4c3bdeb8-efaa-407a-9bea-5f639e284f34-maas-api-tls\") on node \"ip-10-0-129-28.ec2.internal\" DevicePath \"\""
Jul 16 11:42:38.908944 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:42:38.908939    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n9zcz\" (UniqueName: \"kubernetes.io/projected/4c3bdeb8-efaa-407a-9bea-5f639e284f34-kube-api-access-n9zcz\") on node \"ip-10-0-129-28.ec2.internal\" DevicePath \"\""
Jul 16 11:42:38.908944 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:42:38.908949    2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4c3bdeb8-efaa-407a-9bea-5f639e284f34-openshift-service-ca\") on node \"ip-10-0-129-28.ec2.internal\" DevicePath \"\""
Jul 16 11:42:39.076559 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:42:39.076520    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-a47fa3-7b769fd796-mptbl"]
Jul 16 11:42:39.080973 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:42:39.080943    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-a47fa3-7b769fd796-mptbl"]
Jul 16 11:42:39.235698 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:42:39.235656    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="076e3a37-3f46-4197-9447-cf2b1ef9414d" path="/var/lib/kubelet/pods/076e3a37-3f46-4197-9447-cf2b1ef9414d/volumes"
Jul 16 11:42:39.236077 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:42:39.236059    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4c3bdeb8-efaa-407a-9bea-5f639e284f34" path="/var/lib/kubelet/pods/4c3bdeb8-efaa-407a-9bea-5f639e284f34/volumes"
Jul 16 11:42:52.066730 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:42:52.066691    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-98cff6ff4-mw59h_1fdde598-de84-4b7e-8f44-03b0fe2905a5/manager/0.log"
Jul 16 11:42:52.707975 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:42:52.707948    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-58ffff6c5d-xgzpm_a71243ff-abce-4248-8942-fc01f1e7ffab/copy-manifests/0.log"
Jul 16 11:42:52.716738 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:42:52.716700    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-58ffff6c5d-xgzpm_a71243ff-abce-4248-8942-fc01f1e7ffab/manager/0.log"
Jul 16 11:42:53.064088 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:42:53.064011    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-77c48f86df-lp76j_35093d76-01dc-4292-9bff-d38dae1c7680/maas-api/0.log"
Jul 16 11:42:53.178184 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:42:53.178156    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29736675-p49jg_d310be22-325c-4a0a-ba64-60024bd4c6eb/cleanup/0.log"
Jul 16 11:42:53.293607 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:42:53.293575    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29736690-np49l_f333fcd1-055b-4ee5-b0ea-39dd26a58d45/cleanup/0.log"
Jul 16 11:42:55.988295 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:42:55.988267    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4cfrcl_44e90033-3f7d-4432-941d-646531a6f9f4/istio-proxy/0.log"
Jul 16 11:42:56.118096 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:42:56.118065    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-txnpn_af6965b3-3b33-4a0d-bbf8-d18c7e41c21b/discovery/0.log"
Jul 16 11:42:56.871793 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:42:56.871760    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-76cb9bc7db-x2cnj_ec3eb24d-8f3c-40b7-bfe9-a34fccdc20cd/router/0.log"
Jul 16 11:42:57.229649 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:42:57.229615    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-p4mpc_179c766b-106b-4a80-a469-31ceb07ce46b/main/0.log"
Jul 16 11:42:57.585289 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:42:57.585201    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbdtxd5_ae994389-c011-4f3e-b27c-1cded9760559/main/0.log"
Jul 16 11:43:06.750781 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:43:06.750749    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-hlm2s_c88b6ce0-4926-4de9-942e-7f1b30c33e9e/global-pull-secret-syncer/0.log"
Jul 16 11:43:06.923373 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:43:06.923338    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-qxpqt_c24aaddd-b44c-4155-a7c5-fadb281f9b39/konnectivity-agent/0.log"
Jul 16 11:43:06.946459 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:43:06.946428    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-129-28.ec2.internal_43ba1ebae5d7ee21a4751f58c11fde6b/haproxy/0.log"
Jul 16 11:43:14.772823 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:43:14.772791    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-5f7f494dd4-9ztqs_2ae4a146-99bf-422f-982f-0dab385b060b/cluster-monitoring-operator/0.log"
Jul 16 11:43:15.047631 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:43:15.047528    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-6n2hb_44df078b-d929-4693-bce4-5ebde6a53923/node-exporter/0.log"
Jul 16 11:43:15.071656 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:43:15.071631    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-6n2hb_44df078b-d929-4693-bce4-5ebde6a53923/kube-rbac-proxy/0.log"
Jul 16 11:43:15.095859 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:43:15.095829    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-6n2hb_44df078b-d929-4693-bce4-5ebde6a53923/init-textfile/0.log"
Jul 16 11:43:16.369799 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:43:16.369763    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-kdtsf/perf-node-gather-daemonset-l6rzd"]
Jul 16 11:43:16.370212 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:43:16.370169    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="076e3a37-3f46-4197-9447-cf2b1ef9414d" containerName="istio-proxy"
Jul 16 11:43:16.370212 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:43:16.370183    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="076e3a37-3f46-4197-9447-cf2b1ef9414d" containerName="istio-proxy"
Jul 16 11:43:16.370212 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:43:16.370194    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4c3bdeb8-efaa-407a-9bea-5f639e284f34" containerName="maas-api"
Jul 16 11:43:16.370212 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:43:16.370200    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="4c3bdeb8-efaa-407a-9bea-5f639e284f34" containerName="maas-api"
Jul 16 11:43:16.370212 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:43:16.370210    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cbea2e56-7102-4fe4-8817-bef4b535edee" containerName="maas-api"
Jul 16 11:43:16.370212 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:43:16.370216    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="cbea2e56-7102-4fe4-8817-bef4b535edee" containerName="maas-api"
Jul 16 11:43:16.370436 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:43:16.370277    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="076e3a37-3f46-4197-9447-cf2b1ef9414d" containerName="istio-proxy"
Jul 16 11:43:16.370436 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:43:16.370286    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="4c3bdeb8-efaa-407a-9bea-5f639e284f34" containerName="maas-api"
Jul 16 11:43:16.370436 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:43:16.370294    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="cbea2e56-7102-4fe4-8817-bef4b535edee" containerName="maas-api"
Jul 16 11:43:16.373387 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:43:16.373367    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-kdtsf/perf-node-gather-daemonset-l6rzd"
Jul 16 11:43:16.376082 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:43:16.376061    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-kdtsf\"/\"kube-root-ca.crt\""
Jul 16 11:43:16.376981 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:43:16.376959    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-kdtsf\"/\"default-dockercfg-4h66r\""
Jul 16 11:43:16.377114 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:43:16.376996    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-kdtsf\"/\"openshift-service-ca.crt\""
Jul 16 11:43:16.385352 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:43:16.385326    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-kdtsf/perf-node-gather-daemonset-l6rzd"]
Jul 16 11:43:16.425649 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:43:16.425579    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ceea0152-e4a8-456e-beb3-6aad41e8c13e-lib-modules\") pod \"perf-node-gather-daemonset-l6rzd\" (UID: \"ceea0152-e4a8-456e-beb3-6aad41e8c13e\") " pod="openshift-must-gather-kdtsf/perf-node-gather-daemonset-l6rzd"
Jul 16 11:43:16.425840 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:43:16.425691    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s6jv8\" (UniqueName: \"kubernetes.io/projected/ceea0152-e4a8-456e-beb3-6aad41e8c13e-kube-api-access-s6jv8\") pod \"perf-node-gather-daemonset-l6rzd\" (UID: \"ceea0152-e4a8-456e-beb3-6aad41e8c13e\") " pod="openshift-must-gather-kdtsf/perf-node-gather-daemonset-l6rzd"
Jul 16 11:43:16.425840 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:43:16.425744    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/ceea0152-e4a8-456e-beb3-6aad41e8c13e-podres\") pod \"perf-node-gather-daemonset-l6rzd\" (UID: \"ceea0152-e4a8-456e-beb3-6aad41e8c13e\") " pod="openshift-must-gather-kdtsf/perf-node-gather-daemonset-l6rzd"
Jul 16 11:43:16.425840 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:43:16.425778    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/ceea0152-e4a8-456e-beb3-6aad41e8c13e-proc\") pod \"perf-node-gather-daemonset-l6rzd\" (UID: \"ceea0152-e4a8-456e-beb3-6aad41e8c13e\") " pod="openshift-must-gather-kdtsf/perf-node-gather-daemonset-l6rzd"
Jul 16 11:43:16.425951 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:43:16.425903    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ceea0152-e4a8-456e-beb3-6aad41e8c13e-sys\") pod \"perf-node-gather-daemonset-l6rzd\" (UID: \"ceea0152-e4a8-456e-beb3-6aad41e8c13e\") " pod="openshift-must-gather-kdtsf/perf-node-gather-daemonset-l6rzd"
Jul 16 11:43:16.526933 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:43:16.526900    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ceea0152-e4a8-456e-beb3-6aad41e8c13e-sys\") pod \"perf-node-gather-daemonset-l6rzd\" (UID: \"ceea0152-e4a8-456e-beb3-6aad41e8c13e\") " pod="openshift-must-gather-kdtsf/perf-node-gather-daemonset-l6rzd"
Jul 16 11:43:16.527132 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:43:16.526940    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ceea0152-e4a8-456e-beb3-6aad41e8c13e-lib-modules\") pod \"perf-node-gather-daemonset-l6rzd\" (UID: \"ceea0152-e4a8-456e-beb3-6aad41e8c13e\") " pod="openshift-must-gather-kdtsf/perf-node-gather-daemonset-l6rzd"
Jul 16 11:43:16.527132 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:43:16.526960    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s6jv8\" (UniqueName: \"kubernetes.io/projected/ceea0152-e4a8-456e-beb3-6aad41e8c13e-kube-api-access-s6jv8\") pod \"perf-node-gather-daemonset-l6rzd\" (UID: \"ceea0152-e4a8-456e-beb3-6aad41e8c13e\") " pod="openshift-must-gather-kdtsf/perf-node-gather-daemonset-l6rzd"
Jul 16 11:43:16.527132 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:43:16.527057    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ceea0152-e4a8-456e-beb3-6aad41e8c13e-sys\") pod \"perf-node-gather-daemonset-l6rzd\" (UID: \"ceea0152-e4a8-456e-beb3-6aad41e8c13e\") " pod="openshift-must-gather-kdtsf/perf-node-gather-daemonset-l6rzd"
Jul 16 11:43:16.527132 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:43:16.527098    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ceea0152-e4a8-456e-beb3-6aad41e8c13e-lib-modules\") pod \"perf-node-gather-daemonset-l6rzd\" (UID: \"ceea0152-e4a8-456e-beb3-6aad41e8c13e\") " pod="openshift-must-gather-kdtsf/perf-node-gather-daemonset-l6rzd"
Jul 16 11:43:16.527319 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:43:16.527133    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/ceea0152-e4a8-456e-beb3-6aad41e8c13e-podres\") pod \"perf-node-gather-daemonset-l6rzd\" (UID: \"ceea0152-e4a8-456e-beb3-6aad41e8c13e\") " pod="openshift-must-gather-kdtsf/perf-node-gather-daemonset-l6rzd"
Jul 16 11:43:16.527319 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:43:16.527173    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/ceea0152-e4a8-456e-beb3-6aad41e8c13e-proc\") pod \"perf-node-gather-daemonset-l6rzd\" (UID: \"ceea0152-e4a8-456e-beb3-6aad41e8c13e\") " pod="openshift-must-gather-kdtsf/perf-node-gather-daemonset-l6rzd"
Jul 16 11:43:16.527319 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:43:16.527290    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/ceea0152-e4a8-456e-beb3-6aad41e8c13e-proc\") pod \"perf-node-gather-daemonset-l6rzd\" (UID: \"ceea0152-e4a8-456e-beb3-6aad41e8c13e\") " pod="openshift-must-gather-kdtsf/perf-node-gather-daemonset-l6rzd"
Jul 16 11:43:16.527442 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:43:16.527353    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/ceea0152-e4a8-456e-beb3-6aad41e8c13e-podres\") pod \"perf-node-gather-daemonset-l6rzd\" (UID: \"ceea0152-e4a8-456e-beb3-6aad41e8c13e\") " pod="openshift-must-gather-kdtsf/perf-node-gather-daemonset-l6rzd"
Jul 16 11:43:16.543170 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:43:16.543130    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s6jv8\" (UniqueName: \"kubernetes.io/projected/ceea0152-e4a8-456e-beb3-6aad41e8c13e-kube-api-access-s6jv8\") pod \"perf-node-gather-daemonset-l6rzd\" (UID: \"ceea0152-e4a8-456e-beb3-6aad41e8c13e\") " pod="openshift-must-gather-kdtsf/perf-node-gather-daemonset-l6rzd"
Jul 16 11:43:16.683830 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:43:16.683795    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-kdtsf/perf-node-gather-daemonset-l6rzd"
Jul 16 11:43:16.827708 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:43:16.827683    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-kdtsf/perf-node-gather-daemonset-l6rzd"]
Jul 16 11:43:16.829638 ip-10-0-129-28 kubenswrapper[2583]: W0716 11:43:16.829610    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podceea0152_e4a8_456e_beb3_6aad41e8c13e.slice/crio-030dd6233b0743828bab7b02974d4a5e79f8f8c15635a673af6dcd508be9d394 WatchSource:0}: Error finding container 030dd6233b0743828bab7b02974d4a5e79f8f8c15635a673af6dcd508be9d394: Status 404 returned error can't find the container with id 030dd6233b0743828bab7b02974d4a5e79f8f8c15635a673af6dcd508be9d394
Jul 16 11:43:16.936626 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:43:16.936538    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-kdtsf/perf-node-gather-daemonset-l6rzd" event={"ID":"ceea0152-e4a8-456e-beb3-6aad41e8c13e","Type":"ContainerStarted","Data":"030dd6233b0743828bab7b02974d4a5e79f8f8c15635a673af6dcd508be9d394"}
Jul 16 11:43:17.082296 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:43:17.082267    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6cf856758c-vgn2h_9fceac27-6fd9-4091-af74-783b21616981/networking-console-plugin/0.log"
Jul 16 11:43:17.776808 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:43:17.776782    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-9lm5s_f64f15a3-5a9f-4737-bc4f-73d5ec311e41/console-operator/1.log"
Jul 16 11:43:17.783541 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:43:17.783517    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-9lm5s_f64f15a3-5a9f-4737-bc4f-73d5ec311e41/console-operator/2.log"
Jul 16 11:43:17.942947 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:43:17.942909    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-kdtsf/perf-node-gather-daemonset-l6rzd" event={"ID":"ceea0152-e4a8-456e-beb3-6aad41e8c13e","Type":"ContainerStarted","Data":"d558ba20ce988febd96bcc4db4dcb452f3e1bf9e00ee1d091facd036bc751d1c"}
Jul 16 11:43:17.943165 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:43:17.943064    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-kdtsf/perf-node-gather-daemonset-l6rzd"
Jul 16 11:43:18.398898 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:43:18.398864    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-68c9cbb497-755tx_3c28c239-327f-4f45-9bf8-c6cee143cdcb/download-server/0.log"
Jul 16 11:43:19.002521 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:43:19.002489    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-78495597f6-d496q_ca7a1d19-213d-4ce9-8a34-4a93ddcab230/volume-data-source-validator/0.log"
Jul 16 11:43:20.078483 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:43:20.078449    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-5649t_921e629b-d5a3-4bf6-8bad-0131d0a812ea/dns/0.log"
Jul 16 11:43:20.105298 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:43:20.105267    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-5649t_921e629b-d5a3-4bf6-8bad-0131d0a812ea/kube-rbac-proxy/0.log"
Jul 16 11:43:20.294033 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:43:20.293999    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-z5dtv_a3d44785-92d8-4a5e-bc2e-b91f4fa8b509/dns-node-resolver/0.log"
Jul 16 11:43:20.897220 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:43:20.897186    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-6xfs6_0244f4b4-dd39-4870-97be-8378a99c93a2/node-ca/0.log"
Jul 16 11:43:21.981895 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:43:21.981867    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4cfrcl_44e90033-3f7d-4432-941d-646531a6f9f4/istio-proxy/0.log"
Jul 16 11:43:22.091660 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:43:22.091628    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-txnpn_af6965b3-3b33-4a0d-bbf8-d18c7e41c21b/discovery/0.log"
Jul 16 11:43:22.349232 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:43:22.349136    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-76cb9bc7db-x2cnj_ec3eb24d-8f3c-40b7-bfe9-a34fccdc20cd/router/0.log"
Jul 16 11:43:23.027624 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:43:23.027583    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-4mhsn_8dcffe9f-afd4-4e68-9c2a-a6d824df311e/serve-healthcheck-canary/0.log"
Jul 16 11:43:23.616379 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:43:23.616346    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-68d9dd94cf-lx49b_c839c027-e922-48f5-9940-d293b38aef05/insights-operator/0.log"
Jul 16 11:43:23.617127 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:43:23.617111    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-68d9dd94cf-lx49b_c839c027-e922-48f5-9940-d293b38aef05/insights-operator/1.log"
Jul 16 11:43:23.641085 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:43:23.641054    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-kqhbl_19a1fad5-ffe8-43e3-8c58-028cf08b6836/kube-rbac-proxy/0.log"
Jul 16 11:43:23.666864 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:43:23.666839    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-kqhbl_19a1fad5-ffe8-43e3-8c58-028cf08b6836/exporter/0.log"
Jul 16 11:43:23.693371 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:43:23.693341    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-kqhbl_19a1fad5-ffe8-43e3-8c58-028cf08b6836/extractor/0.log"
Jul 16 11:43:23.959559 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:43:23.959532    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-kdtsf/perf-node-gather-daemonset-l6rzd"
Jul 16 11:43:23.980548 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:43:23.980496    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-kdtsf/perf-node-gather-daemonset-l6rzd" podStartSLOduration=7.980480572 podStartE2EDuration="7.980480572s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 11:43:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 11:43:17.977015184 +0000 UTC m=+2390.611236634" watchObservedRunningTime="2026-07-16 11:43:23.980480572 +0000 UTC m=+2396.614702020"
Jul 16 11:43:26.592889 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:43:26.592851    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-98cff6ff4-mw59h_1fdde598-de84-4b7e-8f44-03b0fe2905a5/manager/0.log"
Jul 16 11:43:26.919210 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:43:26.919181    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4cfrcl_44e90033-3f7d-4432-941d-646531a6f9f4/istio-proxy/0.log"
Jul 16 11:43:26.952454 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:43:26.952415    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-58ffff6c5d-xgzpm_a71243ff-abce-4248-8942-fc01f1e7ffab/manager/0.log"
Jul 16 11:43:26.977124 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:43:26.977089    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-58ffff6c5d-xgzpm_a71243ff-abce-4248-8942-fc01f1e7ffab/copy-manifests/0.log"
Jul 16 11:43:27.020377 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:43:27.020345    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-txnpn_af6965b3-3b33-4a0d-bbf8-d18c7e41c21b/discovery/0.log"
Jul 16 11:43:27.215520 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:43:27.215429    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-76cb9bc7db-x2cnj_ec3eb24d-8f3c-40b7-bfe9-a34fccdc20cd/router/0.log"
Jul 16 11:43:28.302195 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:43:28.302163    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-5dbfbcbfc5-564zb_c85bab5b-1491-4e0c-b07a-742c22bb7a9e/manager/0.log"
Jul 16 11:43:28.531470 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:43:28.531443    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4cfrcl_44e90033-3f7d-4432-941d-646531a6f9f4/istio-proxy/0.log"
Jul 16 11:43:28.630632 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:43:28.630585    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-txnpn_af6965b3-3b33-4a0d-bbf8-d18c7e41c21b/discovery/0.log"
Jul 16 11:43:28.845749 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:43:28.845668    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-76cb9bc7db-x2cnj_ec3eb24d-8f3c-40b7-bfe9-a34fccdc20cd/router/0.log"
Jul 16 11:43:29.314064 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:43:29.314034    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-9lm5s_f64f15a3-5a9f-4737-bc4f-73d5ec311e41/console-operator/1.log"
Jul 16 11:43:29.320248 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:43:29.320219    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-9lm5s_f64f15a3-5a9f-4737-bc4f-73d5ec311e41/console-operator/1.log"
Jul 16 11:43:29.321315 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:43:29.321293    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2db6t_69aed169-e8af-429d-99c3-3b20be64e351/ovn-acl-logging/0.log"
Jul 16 11:43:29.329784 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:43:29.329755    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2db6t_69aed169-e8af-429d-99c3-3b20be64e351/ovn-acl-logging/0.log"
Jul 16 11:43:29.453119 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:43:29.453078    2583 scope.go:117] "RemoveContainer" containerID="1370634a9b74c20b21c145e6fb01236a95e2420254a75b508caf3508d57064b6"
Jul 16 11:43:33.401580 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:43:33.401549    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-l5xq6_a5de1249-3e28-4a28-a94a-eee038063ea0/migrator/0.log"
Jul 16 11:43:33.428863 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:43:33.428829    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-l5xq6_a5de1249-3e28-4a28-a94a-eee038063ea0/graceful-termination/0.log"
Jul 16 11:43:33.806690 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:43:33.806576    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-799cb7f958-85d88_e1f38c54-19be-4f6f-872b-39093678a7ad/kube-storage-version-migrator-operator/1.log"
Jul 16 11:43:33.807468 ip-10-0-129-28 kubenswrapper[2583]: I0716 11:43:33.807443    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-799cb7f958-85d88_e1f38c54-19be-4f6f-872b-39093678a7ad/kube-storage-version-migrator-operator/0.log"