Jul 06 18:17:42.826658 ip-10-0-134-113 systemd[1]: Starting Kubernetes Kubelet...
Jul 06 18:17:43.238177 ip-10-0-134-113 kubenswrapper[2580]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 06 18:17:43.238177 ip-10-0-134-113 kubenswrapper[2580]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 06 18:17:43.238177 ip-10-0-134-113 kubenswrapper[2580]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 06 18:17:43.238177 ip-10-0-134-113 kubenswrapper[2580]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 06 18:17:43.238177 ip-10-0-134-113 kubenswrapper[2580]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 06 18:17:43.238177 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.214149    2580 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 06 18:17:43.238177 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.220918    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 06 18:17:43.238177 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.220934    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 06 18:17:43.238177 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.220939    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 06 18:17:43.238177 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.220945    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 06 18:17:43.238177 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.220949    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 06 18:17:43.238177 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.220954    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 06 18:17:43.238177 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.220958    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 06 18:17:43.238177 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.220961    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 06 18:17:43.239011 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.220965    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 06 18:17:43.239011 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.220970    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 06 18:17:43.239011 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.220974    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 06 18:17:43.239011 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.220977    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 06 18:17:43.239011 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.220981    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 06 18:17:43.239011 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.220985    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 06 18:17:43.239011 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.220989    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 06 18:17:43.239011 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.220993    2580 feature_gate.go:328] unrecognized feature gate: Example2
Jul 06 18:17:43.239011 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.220997    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 06 18:17:43.239011 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.221000    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 06 18:17:43.239011 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.221004    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 06 18:17:43.239011 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.221008    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 06 18:17:43.239011 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.221012    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 06 18:17:43.239011 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.221015    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 06 18:17:43.239011 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.221020    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 06 18:17:43.239011 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.221024    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 06 18:17:43.239011 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.221028    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 06 18:17:43.239011 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.221032    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 06 18:17:43.239011 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.221035    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 06 18:17:43.239011 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.221040    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 06 18:17:43.239580 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.221044    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 06 18:17:43.239580 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.221048    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 06 18:17:43.239580 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.221060    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 06 18:17:43.239580 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.221065    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 06 18:17:43.239580 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.221069    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 06 18:17:43.239580 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.221073    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 06 18:17:43.239580 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.221077    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 06 18:17:43.239580 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.221081    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 06 18:17:43.239580 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.221086    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 06 18:17:43.239580 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.221090    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 06 18:17:43.239580 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.221093    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 06 18:17:43.239580 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.221098    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 06 18:17:43.239580 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.221102    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 06 18:17:43.239580 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.221106    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 06 18:17:43.239580 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.221110    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 06 18:17:43.239580 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.221114    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 06 18:17:43.239580 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.221118    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 06 18:17:43.239580 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.221122    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 06 18:17:43.239580 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.221126    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 06 18:17:43.239580 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.221132    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 06 18:17:43.240092 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.221138    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 06 18:17:43.240092 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.221141    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 06 18:17:43.240092 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.221145    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 06 18:17:43.240092 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.221149    2580 feature_gate.go:328] unrecognized feature gate: Example
Jul 06 18:17:43.240092 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.221153    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 06 18:17:43.240092 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.221156    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 06 18:17:43.240092 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.221160    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 06 18:17:43.240092 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.221164    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 06 18:17:43.240092 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.221167    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 06 18:17:43.240092 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.221172    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 06 18:17:43.240092 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.221176    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 06 18:17:43.240092 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.221180    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 06 18:17:43.240092 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.221184    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 06 18:17:43.240092 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.221188    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 06 18:17:43.240092 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.221192    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 06 18:17:43.240092 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.221197    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 06 18:17:43.240092 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.221203    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 06 18:17:43.240092 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.221235    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 06 18:17:43.240092 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.221243    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 06 18:17:43.240691 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.221247    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 06 18:17:43.240691 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.221251    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 06 18:17:43.240691 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.221256    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 06 18:17:43.240691 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.221264    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 06 18:17:43.240691 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.221269    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 06 18:17:43.240691 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.221274    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 06 18:17:43.240691 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.221278    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 06 18:17:43.240691 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.221283    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 06 18:17:43.240691 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.221287    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 06 18:17:43.240691 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.221291    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 06 18:17:43.240691 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.221296    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 06 18:17:43.240691 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.221300    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 06 18:17:43.240691 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.221304    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 06 18:17:43.240691 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.221308    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 06 18:17:43.240691 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.221313    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 06 18:17:43.240691 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.221317    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 06 18:17:43.240691 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.221321    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 06 18:17:43.240691 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.221325    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 06 18:17:43.240691 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.221328    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 06 18:17:43.241183 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.221332    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 06 18:17:43.241183 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.221336    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 06 18:17:43.241183 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.221340    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 06 18:17:43.241183 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.221344    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 06 18:17:43.241183 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.221347    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 06 18:17:43.241183 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.221351    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 06 18:17:43.241183 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.221354    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 06 18:17:43.241183 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.221358    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 06 18:17:43.241183 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.221363    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 06 18:17:43.241183 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.221378    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 06 18:17:43.241183 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.221384    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 06 18:17:43.241183 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.221389    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 06 18:17:43.241183 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.221392    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 06 18:17:43.241183 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.224052    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 06 18:17:43.241183 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.224062    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 06 18:17:43.241183 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.224067    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 06 18:17:43.241183 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.224071    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 06 18:17:43.241183 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.224075    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 06 18:17:43.241183 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.224078    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 06 18:17:43.241183 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.224082    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 06 18:17:43.241748 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.224088    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 06 18:17:43.241748 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.224092    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 06 18:17:43.241748 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.224096    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 06 18:17:43.241748 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.224100    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 06 18:17:43.241748 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.224104    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 06 18:17:43.241748 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.224108    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 06 18:17:43.241748 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.224112    2580 feature_gate.go:328] unrecognized feature gate: Example
Jul 06 18:17:43.241748 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.224115    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 06 18:17:43.241748 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.224119    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 06 18:17:43.241748 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.224123    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 06 18:17:43.241748 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.224127    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 06 18:17:43.241748 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.224131    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 06 18:17:43.241748 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.224138    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 06 18:17:43.241748 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.224141    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 06 18:17:43.241748 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.224145    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 06 18:17:43.241748 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.224149    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 06 18:17:43.241748 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.224152    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 06 18:17:43.241748 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.224156    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 06 18:17:43.241748 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.224160    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 06 18:17:43.241748 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.224164    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 06 18:17:43.242309 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.224168    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 06 18:17:43.242309 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.224172    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 06 18:17:43.242309 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.224176    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 06 18:17:43.242309 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.224180    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 06 18:17:43.242309 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.224185    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 06 18:17:43.242309 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.224189    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 06 18:17:43.242309 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.224193    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 06 18:17:43.242309 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.224197    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 06 18:17:43.242309 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.224201    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 06 18:17:43.242309 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.224205    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 06 18:17:43.242309 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.224209    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 06 18:17:43.242309 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.224212    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 06 18:17:43.242309 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.224216    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 06 18:17:43.242309 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.224220    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 06 18:17:43.242309 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.224223    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 06 18:17:43.242309 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.224227    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 06 18:17:43.242309 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.224231    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 06 18:17:43.242309 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.224236    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 06 18:17:43.242309 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.224240    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 06 18:17:43.242309 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.224244    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 06 18:17:43.242909 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.224248    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 06 18:17:43.242909 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.224252    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 06 18:17:43.242909 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.224255    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 06 18:17:43.242909 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.224260    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 06 18:17:43.242909 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.224265    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 06 18:17:43.242909 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.224269    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 06 18:17:43.242909 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.224274    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 06 18:17:43.242909 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.224278    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 06 18:17:43.242909 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.224284    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 06 18:17:43.242909 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.224290    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 06 18:17:43.242909 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.224294    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 06 18:17:43.242909 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.224298    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 06 18:17:43.242909 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.224302    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 06 18:17:43.242909 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.224306    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 06 18:17:43.242909 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.224310    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 06 18:17:43.242909 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.224314    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 06 18:17:43.242909 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.224318    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 06 18:17:43.242909 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.224323    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 06 18:17:43.242909 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.224326    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 06 18:17:43.243427 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.224330    2580 feature_gate.go:328] unrecognized feature gate: Example2
Jul 06 18:17:43.243427 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.224334    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 06 18:17:43.243427 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.224338    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 06 18:17:43.243427 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.224342    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 06 18:17:43.243427 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.224346    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 06 18:17:43.243427 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.224354    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 06 18:17:43.243427 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.224358    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 06 18:17:43.243427 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.224363    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 06 18:17:43.243427 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.224382    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 06 18:17:43.243427 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.224386    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 06 18:17:43.243427 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.224390    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 06 18:17:43.243427 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.224394    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 06 18:17:43.243427 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.224397    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 06 18:17:43.243427 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.224403    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 06 18:17:43.243427 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.224408    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 06 18:17:43.243427 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.224413    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 06 18:17:43.243427 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.224417    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 06 18:17:43.243427 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.224422    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 06 18:17:43.243427 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.224426    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 06 18:17:43.243964 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.224430    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 06 18:17:43.243964 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.224434    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 06 18:17:43.243964 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.224438    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 06 18:17:43.243964 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.224442    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 06 18:17:43.243964 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.224446    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 06 18:17:43.243964 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.224450    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 06 18:17:43.243964 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.224453    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 06 18:17:43.243964 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.224457    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 06 18:17:43.243964 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.224461    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 06 18:17:43.243964 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.224465    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 06 18:17:43.243964 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.224469    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 06 18:17:43.243964 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.224473    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 06 18:17:43.243964 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.224476    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 06 18:17:43.243964 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.224480    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 06 18:17:43.243964 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.225059    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 06 18:17:43.243964 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.225066    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 06 18:17:43.243964 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.225071    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 06 18:17:43.243964 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.225075    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 06 18:17:43.243964 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.225079    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 06 18:17:43.243964 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.225083    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 06 18:17:43.244509 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.225088    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 06 18:17:43.244509 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.225092    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 06 18:17:43.244509 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.225096    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 06 18:17:43.244509 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.225100    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 06 18:17:43.244509 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.225104    2580 feature_gate.go:328] unrecognized feature gate: Example
Jul 06 18:17:43.244509 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.225108    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 06 18:17:43.244509 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.225112    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 06 18:17:43.244509 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.225116    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 06 18:17:43.244509 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.225120    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 06 18:17:43.244509 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.225126    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 06 18:17:43.244509 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.225132    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 06 18:17:43.244509 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.225136    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 06 18:17:43.244509 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.225140    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 06 18:17:43.244509 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.225144    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 06 18:17:43.244509 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.225148    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 06 18:17:43.244509 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.225151    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 06 18:17:43.244509 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.225155    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 06 18:17:43.244509 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.225159    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 06 18:17:43.244509 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.225163    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 06 18:17:43.245052 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.225167    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 06 18:17:43.245052 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.225170    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 06 18:17:43.245052 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.225174    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 06 18:17:43.245052 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.225178    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 06 18:17:43.245052 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.225182    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 06 18:17:43.245052 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.225187    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 06 18:17:43.245052 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.225193    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 06 18:17:43.245052 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.225199    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 06 18:17:43.245052 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.225203    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 06 18:17:43.245052 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.225206    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 06 18:17:43.245052 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.225210    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 06 18:17:43.245052 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.225214    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 06 18:17:43.245052 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.225218    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 06 18:17:43.245052 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.225222    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 06 18:17:43.245052 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.225226    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 06 18:17:43.245052 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.225229    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 06 18:17:43.245052 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.225233    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 06 18:17:43.245052 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.225237    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 06 18:17:43.245052 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.225240    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 06 18:17:43.245586 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.225244    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 06 18:17:43.245586 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.225249    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 06 18:17:43.245586 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.225252    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 06 18:17:43.245586 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.225256    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 06 18:17:43.245586 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.225262    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 06 18:17:43.245586 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.225266    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 06 18:17:43.245586 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.225270    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 06 18:17:43.245586 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.225274    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 06 18:17:43.245586 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.225277    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 06 18:17:43.245586 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.225281    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 06 18:17:43.245586 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.225285    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 06 18:17:43.245586 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.225289    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 06 18:17:43.245586 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.225294    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 06 18:17:43.245586 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.225298    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 06 18:17:43.245586 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.225301    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 06 18:17:43.245586 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.225305    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 06 18:17:43.245586 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.225308    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 06 18:17:43.245586 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.225312    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 06 18:17:43.245586 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.225317    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 06 18:17:43.245586 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.225320    2580 feature_gate.go:328] unrecognized feature gate: Example2
Jul 06 18:17:43.246175 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.225326    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 06 18:17:43.246175 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.225330    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 06 18:17:43.246175 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.225334    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 06 18:17:43.246175 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.225338    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 06 18:17:43.246175 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.225341    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 06 18:17:43.246175 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.225345    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 06 18:17:43.246175 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.225351    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 06 18:17:43.246175 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.225354    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 06 18:17:43.246175 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.225358    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 06 18:17:43.246175 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.225361    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 06 18:17:43.246175 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.225380    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 06 18:17:43.246175 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.225384    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 06 18:17:43.246175 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.225387    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 06 18:17:43.246175 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.225391    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 06 18:17:43.246175 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.225395    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 06 18:17:43.246175 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.225399    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 06 18:17:43.246175 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.225404    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 06 18:17:43.246175 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.225407    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 06 18:17:43.246175 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.225411    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 06 18:17:43.246175 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.225415    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 06 18:17:43.248845 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.225419    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 06 18:17:43.248845 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.225423    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 06 18:17:43.248845 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.225427    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 06 18:17:43.248845 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.225430    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 06 18:17:43.248845 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.225434    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 06 18:17:43.248845 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.225438    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 06 18:17:43.248845 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.225442    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 06 18:17:43.248845 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.225448    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 06 18:17:43.248845 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.225452    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 06 18:17:43.248845 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.225456    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 06 18:17:43.248845 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.225461    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 06 18:17:43.248845 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.225465    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 06 18:17:43.248845 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.225471    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 06 18:17:43.248845 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.225475    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 06 18:17:43.248845 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.225479    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 06 18:17:43.248845 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.225563    2580 flags.go:64] FLAG: --address="0.0.0.0"
Jul 06 18:17:43.248845 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.225573    2580 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 06 18:17:43.248845 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.225581    2580 flags.go:64] FLAG: --anonymous-auth="true"
Jul 06 18:17:43.248845 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.225587    2580 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 06 18:17:43.248845 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.225592    2580 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 06 18:17:43.248845 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.225597    2580 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 06 18:17:43.249636 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.225603    2580 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 06 18:17:43.249636 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.225609    2580 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 06 18:17:43.249636 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.225614    2580 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 06 18:17:43.249636 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.225619    2580 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 06 18:17:43.249636 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.225624    2580 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 06 18:17:43.249636 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.225629    2580 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 06 18:17:43.249636 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.225634    2580 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 06 18:17:43.249636 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.225640    2580 flags.go:64] FLAG: --cgroup-root=""
Jul 06 18:17:43.249636 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.225644    2580 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 06 18:17:43.249636 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.225648    2580 flags.go:64] FLAG: --client-ca-file=""
Jul 06 18:17:43.249636 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.225654    2580 flags.go:64] FLAG: --cloud-provider="external"
Jul 06 18:17:43.249636 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.225658    2580 flags.go:64] FLAG: --cluster-dns="[]"
Jul 06 18:17:43.249636 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.225665    2580 flags.go:64] FLAG: --cluster-domain=""
Jul 06 18:17:43.249636 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.225669    2580 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 06 18:17:43.249636 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.225673    2580 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 06 18:17:43.249636 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.225678    2580 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 06 18:17:43.249636 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.225683    2580 flags.go:64] FLAG: --container-log-max-files="5"
Jul 06 18:17:43.249636 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.225687    2580 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 06 18:17:43.249636 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.225692    2580 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 06 18:17:43.249636 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.225696    2580 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 06 18:17:43.249636 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.225701    2580 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 06 18:17:43.249636 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.225705    2580 flags.go:64] FLAG: --contention-profiling="false"
Jul 06 18:17:43.249636 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.225710    2580 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 06 18:17:43.249636 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.225716    2580 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 06 18:17:43.250240 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.225721    2580 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 06 18:17:43.250240 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.225725    2580 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 06 18:17:43.250240 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.225731    2580 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 06 18:17:43.250240 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.225736    2580 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 06 18:17:43.250240 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.225740    2580 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 06 18:17:43.250240 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.225744    2580 flags.go:64] FLAG: --enable-load-reader="false"
Jul 06 18:17:43.250240 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.225748    2580 flags.go:64] FLAG: --enable-server="true"
Jul 06 18:17:43.250240 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.225752    2580 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 06 18:17:43.250240 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.225758    2580 flags.go:64] FLAG: --event-burst="100"
Jul 06 18:17:43.250240 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.225763    2580 flags.go:64] FLAG: --event-qps="50"
Jul 06 18:17:43.250240 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.225767    2580 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 06 18:17:43.250240 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.225771    2580 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 06 18:17:43.250240 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.225775    2580 flags.go:64] FLAG: --eviction-hard=""
Jul 06 18:17:43.250240 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.225780    2580 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 06 18:17:43.250240 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.225784    2580 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 06 18:17:43.250240 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.225790    2580 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 06 18:17:43.250240 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.225795    2580 flags.go:64] FLAG: --eviction-soft=""
Jul 06 18:17:43.250240 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.225799    2580 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 06 18:17:43.250240 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.225803    2580 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 06 18:17:43.250240 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.225807    2580 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 06 18:17:43.250240 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.225812    2580 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 06 18:17:43.250240 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.225816    2580 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 06 18:17:43.250240 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.225820    2580 flags.go:64] FLAG: --fail-swap-on="true"
Jul 06 18:17:43.250240 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.225825    2580 flags.go:64] FLAG: --feature-gates=""
Jul 06 18:17:43.250240 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.225830    2580 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 06 18:17:43.250980 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.225834    2580 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 06 18:17:43.250980 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.225839    2580 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 06 18:17:43.250980 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.225844    2580 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 06 18:17:43.250980 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.225848    2580 flags.go:64] FLAG: --healthz-port="10248"
Jul 06 18:17:43.250980 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.225852    2580 flags.go:64] FLAG: --help="false"
Jul 06 18:17:43.250980 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.225856    2580 flags.go:64] FLAG: --hostname-override="ip-10-0-134-113.ec2.internal"
Jul 06 18:17:43.250980 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.225863    2580 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 06 18:17:43.250980 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.225868    2580 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 06 18:17:43.250980 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.225872    2580 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 06 18:17:43.250980 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.225877    2580 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 06 18:17:43.250980 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.225882    2580 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 06 18:17:43.250980 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.225886    2580 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 06 18:17:43.250980 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.225890    2580 flags.go:64] FLAG: --image-service-endpoint=""
Jul 06 18:17:43.250980 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.225894    2580 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 06 18:17:43.250980 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.225898    2580 flags.go:64] FLAG: --kube-api-burst="100"
Jul 06 18:17:43.250980 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.225903    2580 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 06 18:17:43.250980 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.225908    2580 flags.go:64] FLAG: --kube-api-qps="50"
Jul 06 18:17:43.250980 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.225912    2580 flags.go:64] FLAG: --kube-reserved=""
Jul 06 18:17:43.250980 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.225916    2580 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 06 18:17:43.250980 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.225920    2580 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 06 18:17:43.250980 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.225924    2580 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 06 18:17:43.250980 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.225928    2580 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 06 18:17:43.250980 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.225934    2580 flags.go:64] FLAG: --lock-file=""
Jul 06 18:17:43.250980 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.225938    2580 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 06 18:17:43.437754 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.225942    2580 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 06 18:17:43.437754 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.225947    2580 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 06 18:17:43.437754 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.225953    2580 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 06 18:17:43.437754 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.225958    2580 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 06 18:17:43.437754 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.225962    2580 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 06 18:17:43.437754 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.225966    2580 flags.go:64] FLAG: --logging-format="text"
Jul 06 18:17:43.437754 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.225970    2580 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 06 18:17:43.437754 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.225975    2580 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 06 18:17:43.437754 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.225979    2580 flags.go:64] FLAG: --manifest-url=""
Jul 06 18:17:43.437754 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.225983    2580 flags.go:64] FLAG: --manifest-url-header=""
Jul 06 18:17:43.437754 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.225989    2580 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 06 18:17:43.437754 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.225993    2580 flags.go:64] FLAG: --max-open-files="1000000"
Jul 06 18:17:43.437754 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.225999    2580 flags.go:64] FLAG: --max-pods="110"
Jul 06 18:17:43.437754 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.226003    2580 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 06 18:17:43.437754 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.226009    2580 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 06 18:17:43.437754 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.226013    2580 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 06 18:17:43.437754 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.226029    2580 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 06 18:17:43.437754 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.226034    2580 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 06 18:17:43.437754 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.226038    2580 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 06 18:17:43.437754 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.226042    2580 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 06 18:17:43.437754 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.226052    2580 flags.go:64] FLAG: --node-status-max-images="50"
Jul 06 18:17:43.437754 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.226056    2580 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 06 18:17:43.437754 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.226061    2580 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 06 18:17:43.437754 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.226065    2580 flags.go:64] FLAG: --pod-cidr=""
Jul 06 18:17:43.292420 ip-10-0-134-113 systemd[1]: Started Kubernetes Kubelet.
Jul 06 18:17:45.347909 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.226070    2580 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5"
Jul 06 18:17:45.347909 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.226078    2580 flags.go:64] FLAG: --pod-manifest-path=""
Jul 06 18:17:45.347909 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.226083    2580 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 06 18:17:45.347909 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.226087    2580 flags.go:64] FLAG: --pods-per-core="0"
Jul 06 18:17:45.347909 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.226091    2580 flags.go:64] FLAG: --port="10250"
Jul 06 18:17:45.347909 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.226096    2580 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 06 18:17:45.347909 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.226100    2580 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-064092b4b327069dc"
Jul 06 18:17:45.347909 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.226106    2580 flags.go:64] FLAG: --qos-reserved=""
Jul 06 18:17:45.347909 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.226110    2580 flags.go:64] FLAG: --read-only-port="10255"
Jul 06 18:17:45.347909 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.226115    2580 flags.go:64] FLAG: --register-node="true"
Jul 06 18:17:45.347909 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.226120    2580 flags.go:64] FLAG: --register-with-taints=""
Jul 06 18:17:45.347909 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.226125    2580 flags.go:64] FLAG: --registry-burst="10"
Jul 06 18:17:45.347909 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.226129    2580 flags.go:64] FLAG: --registry-qps="5"
Jul 06 18:17:45.347909 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.226134    2580 flags.go:64] FLAG: --reserved-cpus=""
Jul 06 18:17:45.347909 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.226138    2580 flags.go:64] FLAG: --reserved-memory=""
Jul 06 18:17:45.347909 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.226143    2580 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 06 18:17:45.347909 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.226147    2580 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 06 18:17:45.347909 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.226152    2580 flags.go:64] FLAG: --rotate-certificates="false"
Jul 06 18:17:45.347909 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.226156    2580 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 06 18:17:45.347909 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.226160    2580 flags.go:64] FLAG: --runonce="false"
Jul 06 18:17:45.347909 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.226164    2580 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 06 18:17:45.347909 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.226169    2580 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 06 18:17:45.347909 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.226174    2580 flags.go:64] FLAG: --seccomp-default="false"
Jul 06 18:17:45.347909 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.226178    2580 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 06 18:17:45.347909 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.226182    2580 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 06 18:17:45.349684 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.226187    2580 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 06 18:17:45.349684 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.226191    2580 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 06 18:17:45.349684 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.226196    2580 flags.go:64] FLAG: --storage-driver-password="root"
Jul 06 18:17:45.349684 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.226199    2580 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 06 18:17:45.349684 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.226203    2580 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 06 18:17:45.349684 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.226207    2580 flags.go:64] FLAG: --storage-driver-user="root"
Jul 06 18:17:45.349684 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.226211    2580 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 06 18:17:45.349684 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.226216    2580 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 06 18:17:45.349684 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.226220    2580 flags.go:64] FLAG: --system-cgroups=""
Jul 06 18:17:45.349684 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.226225    2580 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 06 18:17:45.349684 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.226232    2580 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 06 18:17:45.349684 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.226236    2580 flags.go:64] FLAG: --tls-cert-file=""
Jul 06 18:17:45.349684 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.226239    2580 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 06 18:17:45.349684 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.226246    2580 flags.go:64] FLAG: --tls-min-version=""
Jul 06 18:17:45.349684 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.226252    2580 flags.go:64] FLAG: --tls-private-key-file=""
Jul 06 18:17:45.349684 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.226255    2580 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 06 18:17:45.349684 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.226260    2580 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 06 18:17:45.349684 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.226264    2580 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 06 18:17:45.349684 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.226268    2580 flags.go:64] FLAG: --v="2"
Jul 06 18:17:45.349684 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.226274    2580 flags.go:64] FLAG: --version="false"
Jul 06 18:17:45.349684 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.226279    2580 flags.go:64] FLAG: --vmodule=""
Jul 06 18:17:45.349684 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.226285    2580 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 06 18:17:45.349684 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.226289    2580 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 06 18:17:45.349684 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.226492    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 06 18:17:45.349684 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.226501    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 06 18:17:45.350611 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.226506    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 06 18:17:45.350611 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.226511    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 06 18:17:45.350611 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.226517    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 06 18:17:45.350611 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.226522    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 06 18:17:45.350611 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.226538    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 06 18:17:45.350611 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.226543    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 06 18:17:45.350611 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.226547    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 06 18:17:45.350611 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.226551    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 06 18:17:45.350611 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.226556    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 06 18:17:45.350611 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.226560    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 06 18:17:45.350611 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.226564    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 06 18:17:45.350611 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.226568    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 06 18:17:45.350611 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.226572    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 06 18:17:45.350611 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.226575    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 06 18:17:45.350611 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.226579    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 06 18:17:45.350611 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.226583    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 06 18:17:45.350611 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.226587    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 06 18:17:45.350611 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.226590    2580 feature_gate.go:328] unrecognized feature gate: Example2
Jul 06 18:17:45.350611 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.226594    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 06 18:17:45.351255 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.226598    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 06 18:17:45.351255 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.226602    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 06 18:17:45.351255 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.226607    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 06 18:17:45.351255 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.226611    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 06 18:17:45.351255 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.226615    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 06 18:17:45.351255 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.226618    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 06 18:17:45.351255 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.226622    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 06 18:17:45.351255 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.226626    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 06 18:17:45.351255 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.226632    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 06 18:17:45.351255 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.226636    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 06 18:17:45.351255 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.226641    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 06 18:17:45.351255 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.226645    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 06 18:17:45.351255 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.226650    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 06 18:17:45.351255 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.226653    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 06 18:17:45.351255 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.226658    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 06 18:17:45.351255 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.226662    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 06 18:17:45.351255 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.226665    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 06 18:17:45.351255 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.226670    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 06 18:17:45.351255 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.226674    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 06 18:17:45.351864 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.226678    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 06 18:17:45.351864 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.226681    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 06 18:17:45.351864 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.226685    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 06 18:17:45.351864 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.226752    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 06 18:17:45.351864 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.227009    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 06 18:17:45.351864 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.227014    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 06 18:17:45.351864 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.227019    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 06 18:17:45.351864 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.227024    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 06 18:17:45.351864 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.227028    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 06 18:17:45.351864 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.227032    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 06 18:17:45.351864 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.227036    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 06 18:17:45.351864 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.227040    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 06 18:17:45.351864 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.227044    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 06 18:17:45.351864 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.227048    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 06 18:17:45.351864 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.227053    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 06 18:17:45.351864 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.227065    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 06 18:17:45.351864 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.227071    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 06 18:17:45.351864 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.227076    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 06 18:17:45.351864 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.227080    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 06 18:17:45.351864 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.227085    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 06 18:17:45.352555 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.227127    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 06 18:17:45.352555 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.227132    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 06 18:17:45.352555 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.227136    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 06 18:17:45.352555 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.227140    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 06 18:17:45.352555 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.227144    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 06 18:17:45.352555 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.227149    2580 feature_gate.go:328] unrecognized feature gate: Example
Jul 06 18:17:45.352555 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.227153    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 06 18:17:45.352555 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.227157    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 06 18:17:45.352555 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.227166    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 06 18:17:45.352555 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.227171    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 06 18:17:45.352555 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.227177    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 06 18:17:45.352555 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.227181    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 06 18:17:45.352555 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.227186    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 06 18:17:45.352555 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.227190    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 06 18:17:45.352555 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.227194    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 06 18:17:45.352555 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.227198    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 06 18:17:45.352555 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.227228    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 06 18:17:45.352555 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.227233    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 06 18:17:45.352555 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.227237    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 06 18:17:45.352555 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.227241    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 06 18:17:45.353261 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.227249    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 06 18:17:45.353261 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.227253    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 06 18:17:45.353261 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.227258    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 06 18:17:45.353261 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.227262    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 06 18:17:45.353261 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.227266    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 06 18:17:45.353261 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.227270    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 06 18:17:45.353261 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.227274    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 06 18:17:45.353261 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.227280    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 06 18:17:45.353261 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.227283    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 06 18:17:45.353261 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.227287    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 06 18:17:45.353261 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.227292    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 06 18:17:45.353261 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.227295    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 06 18:17:45.353261 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.227299    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 06 18:17:45.353261 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.227307    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 06 18:17:45.353261 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.227311    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 06 18:17:45.353261 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.227316    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 06 18:17:45.353261 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.227320    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 06 18:17:45.353261 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.227324    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 06 18:17:45.353261 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.227328    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 06 18:17:45.353825 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.227948    2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 06 18:17:45.353825 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.236655    2580 server.go:529] "Kubelet version" kubeletVersion="v1.34.8"
Jul 06 18:17:45.353825 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.236670    2580 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 06 18:17:45.353825 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.237427    2580 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 06 18:17:45.353825 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.237434    2580 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 06 18:17:45.353825 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.237494    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 06 18:17:45.353825 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.237500    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 06 18:17:45.353825 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.237503    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 06 18:17:45.353825 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.237506    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 06 18:17:45.353825 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.237510    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 06 18:17:45.353825 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.237515    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 06 18:17:45.353825 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.237518    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 06 18:17:45.353825 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.237521    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 06 18:17:45.353825 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.237524    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 06 18:17:45.595582 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.237526    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 06 18:17:45.595582 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.237529    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 06 18:17:45.595582 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.237532    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 06 18:17:45.595582 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.237535    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 06 18:17:45.595582 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.237538    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 06 18:17:45.595582 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.237540    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 06 18:17:45.595582 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.237543    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 06 18:17:45.595582 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.237545    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 06 18:17:45.595582 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.237548    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 06 18:17:45.595582 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.237551    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 06 18:17:45.595582 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.237553    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 06 18:17:45.595582 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.237556    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 06 18:17:45.595582 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.237559    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 06 18:17:45.595582 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.237561    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 06 18:17:45.595582 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.237564    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 06 18:17:45.595582 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.237566    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 06 18:17:45.595582 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.237570    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 06 18:17:45.595582 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.237573    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 06 18:17:45.595582 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.237575    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 06 18:17:45.596277 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.237578    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 06 18:17:45.596277 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.237581    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 06 18:17:45.596277 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.237583    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 06 18:17:45.596277 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.237586    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 06 18:17:45.596277 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.237589    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 06 18:17:45.596277 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.237591    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 06 18:17:45.596277 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.237594    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 06 18:17:45.596277 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.237596    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 06 18:17:45.596277 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.237599    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 06 18:17:45.596277 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.237604    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 06 18:17:45.596277 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.237607    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 06 18:17:45.596277 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.237609    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 06 18:17:45.596277 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.237612    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 06 18:17:45.596277 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.237615    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 06 18:17:45.596277 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.237617    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 06 18:17:45.596277 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.237620    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 06 18:17:45.596277 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.237622    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 06 18:17:45.596277 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.237625    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 06 18:17:45.596277 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.237627    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 06 18:17:45.596277 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.237630    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 06 18:17:45.596917 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.237633    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 06 18:17:45.596917 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.237636    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 06 18:17:45.596917 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.237638    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 06 18:17:45.596917 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.237641    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 06 18:17:45.596917 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.237643    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 06 18:17:45.596917 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.237648    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 06 18:17:45.596917 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.237650    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 06 18:17:45.596917 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.237654    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 06 18:17:45.596917 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.237656    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 06 18:17:45.596917 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.237659    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 06 18:17:45.596917 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.237662    2580 feature_gate.go:328] unrecognized feature gate: Example2
Jul 06 18:17:45.596917 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.237664    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 06 18:17:45.596917 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.237667    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 06 18:17:45.596917 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.237669    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 06 18:17:45.596917 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.237672    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 06 18:17:45.596917 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.237674    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 06 18:17:45.596917 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.237677    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 06 18:17:45.596917 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.237679    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 06 18:17:45.596917 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.237682    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 06 18:17:45.596917 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.237684    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 06 18:17:45.597535 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.237687    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 06 18:17:45.597535 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.237691    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 06 18:17:45.597535 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.237693    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 06 18:17:45.597535 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.237696    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 06 18:17:45.597535 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.237699    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 06 18:17:45.597535 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.237701    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 06 18:17:45.597535 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.237704    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 06 18:17:45.597535 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.237706    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 06 18:17:45.597535 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.237709    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 06 18:17:45.597535 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.237711    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 06 18:17:45.597535 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.237714    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 06 18:17:45.597535 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.237716    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 06 18:17:45.597535 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.237719    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 06 18:17:45.597535 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.237721    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 06 18:17:45.597535 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.237724    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 06 18:17:45.597535 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.237726    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 06 18:17:45.597535 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.237729    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 06 18:17:45.597535 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.237734    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 06 18:17:45.597535 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.237737    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 06 18:17:45.598159 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.237741    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 06 18:17:45.598159 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.237744    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 06 18:17:45.598159 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.237747    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 06 18:17:45.598159 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.237750    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 06 18:17:45.598159 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.237753    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 06 18:17:45.598159 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.237756    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 06 18:17:45.598159 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.237758    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 06 18:17:45.598159 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.237762    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 06 18:17:45.598159 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.237766    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 06 18:17:45.598159 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.237769    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 06 18:17:45.598159 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.237772    2580 feature_gate.go:328] unrecognized feature gate: Example
Jul 06 18:17:45.598159 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.237774    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 06 18:17:45.598159 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.237779    2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 06 18:17:45.598159 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.237903    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 06 18:17:45.598159 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.237908    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 06 18:17:45.598666 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.237913    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 06 18:17:45.598666 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.237916    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 06 18:17:45.598666 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.237919    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 06 18:17:45.598666 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.237921    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 06 18:17:45.598666 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.237924    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 06 18:17:45.598666 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.237927    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 06 18:17:45.598666 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.237930    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 06 18:17:45.598666 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.237932    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 06 18:17:45.598666 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.237935    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 06 18:17:45.598666 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.237938    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 06 18:17:45.598666 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.237940    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 06 18:17:45.598666 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.237943    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 06 18:17:45.598666 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.237946    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 06 18:17:45.598666 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.237948    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 06 18:17:45.598666 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.237951    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 06 18:17:45.598666 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.237954    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 06 18:17:45.598666 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.237957    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 06 18:17:45.598666 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.237960    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 06 18:17:45.598666 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.237965    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 06 18:17:45.599256 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.237969    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 06 18:17:45.599256 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.237972    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 06 18:17:45.599256 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.237975    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 06 18:17:45.599256 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.237977    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 06 18:17:45.599256 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.237980    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 06 18:17:45.599256 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.237983    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 06 18:17:45.599256 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.237985    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 06 18:17:45.599256 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.237988    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 06 18:17:45.599256 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.237990    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 06 18:17:45.599256 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.237993    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 06 18:17:45.599256 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.237995    2580 feature_gate.go:328] unrecognized feature gate: Example
Jul 06 18:17:45.599256 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.237998    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 06 18:17:45.599256 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.238000    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 06 18:17:45.599256 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.238005    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 06 18:17:45.599256 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.238007    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 06 18:17:45.599256 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.238010    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 06 18:17:45.599256 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.238012    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 06 18:17:45.599256 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.238015    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 06 18:17:45.599256 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.238018    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 06 18:17:45.599256 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.238021    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 06 18:17:45.599795 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.238023    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 06 18:17:45.599795 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.238026    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 06 18:17:45.599795 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.238028    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 06 18:17:45.599795 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.238031    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 06 18:17:45.599795 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.238033    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 06 18:17:45.599795 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.238036    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 06 18:17:45.599795 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.238038    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 06 18:17:45.599795 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.238041    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 06 18:17:45.599795 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.238044    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 06 18:17:45.599795 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.238047    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 06 18:17:45.599795 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.238049    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 06 18:17:45.599795 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.238052    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 06 18:17:45.599795 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.238055    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 06 18:17:45.599795 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.238058    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 06 18:17:45.599795 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.238061    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 06 18:17:45.599795 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.238064    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 06 18:17:45.599795 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.238066    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 06 18:17:45.599795 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.238068    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 06 18:17:45.599795 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.238071    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 06 18:17:45.600352 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.238075    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 06 18:17:45.600352 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.238078    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 06 18:17:45.600352 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.238081    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 06 18:17:45.600352 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.238083    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 06 18:17:45.600352 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.238086    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 06 18:17:45.600352 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.238089    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 06 18:17:45.600352 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.238093    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 06 18:17:45.600352 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.238096    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 06 18:17:45.600352 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.238099    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 06 18:17:45.600352 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.238101    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 06 18:17:45.600352 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.238104    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 06 18:17:45.600352 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.238106    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 06 18:17:45.600352 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.238109    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 06 18:17:45.600352 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.238111    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 06 18:17:45.600352 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.238114    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 06 18:17:45.600352 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.238117    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 06 18:17:45.600352 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.238119    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 06 18:17:45.600352 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.238121    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 06 18:17:45.600352 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.238124    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 06 18:17:45.600352 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.238127    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 06 18:17:45.600892 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.238129    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 06 18:17:45.600892 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.238135    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 06 18:17:45.600892 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.238138    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 06 18:17:45.600892 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.238140    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 06 18:17:45.600892 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.238142    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 06 18:17:45.600892 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.238145    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 06 18:17:45.600892 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.238164    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 06 18:17:45.600892 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.238167    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 06 18:17:45.600892 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.238169    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 06 18:17:45.600892 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.238172    2580 feature_gate.go:328] unrecognized feature gate: Example2
Jul 06 18:17:45.600892 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.238175    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 06 18:17:45.600892 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.238177    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 06 18:17:45.600892 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.238180    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 06 18:17:45.600892 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.238182    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 06 18:17:45.600892 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.238186    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 06 18:17:45.600892 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.238189    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 06 18:17:45.600892 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.238192    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 06 18:17:45.600892 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.238195    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 06 18:17:45.600892 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:43.238198    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 06 18:17:45.601469 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.238203    2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 06 18:17:45.601469 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.238363    2580 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 06 18:17:45.601469 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.240385    2580 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 06 18:17:45.601469 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.241744    2580 server.go:1019] "Starting client certificate rotation"
Jul 06 18:17:45.601469 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.241843    2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 06 18:17:45.601469 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.241880    2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 06 18:17:45.601469 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.244223    2580 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 06 18:17:45.601469 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.245392    2580 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 06 18:17:45.601469 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.257125    2580 log.go:25] "Validated CRI v1 runtime API"
Jul 06 18:17:45.601469 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.259261    2580 log.go:25] "Validated CRI v1 image API"
Jul 06 18:17:45.601469 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.260192    2580 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 06 18:17:45.601469 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.266515    2580 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 8e5fe57e-428f-46ce-9736-da331775c6cd:/dev/nvme0n1p3 d3d7a749-efae-4dcb-be3f-5d9c9ecce27c:/dev/nvme0n1p4]
Jul 06 18:17:45.601816 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.266538    2580 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 06 18:17:45.601816 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.272072    2580 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 06 18:17:45.602306 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.273476    2580 manager.go:216] Machine: {Timestamp:2026-07-06 18:17:43.272300238 +0000 UTC m=+0.324344830 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3132520 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec27c7a84e80f992958801c762c9ed45 SystemUUID:ec27c7a8-4e80-f992-9588-01c762c9ed45 BootID:45827980-c1f4-46ab-a217-f960deda8ce4 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ca:48:70:91:fd Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ca:48:70:91:fd Speed:0 Mtu:9001} {Name:ovs-system MacAddress:c6:fb:08:b4:59:de Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 06 18:17:45.602306 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.273572    2580 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 06 18:17:45.602306 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.273684    2580 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 06 18:17:45.602306 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.275301    2580 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 06 18:17:45.602306 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.275327    2580 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-134-113.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 06 18:17:45.602306 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.275914    2580 topology_manager.go:138] "Creating topology manager with none policy"
Jul 06 18:17:45.602306 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.275924    2580 container_manager_linux.go:309] "Creating device plugin manager"
Jul 06 18:17:45.602306 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.275931    2580 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 06 18:17:45.602306 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.276592    2580 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 06 18:17:45.602306 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.277759    2580 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 06 18:17:45.602306 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.280526    2580 state_mem.go:36] "Initialized new in-memory state store"
Jul 06 18:17:45.602306 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.280628    2580 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 06 18:17:45.602306 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.280656    2580 kubelet.go:486] "Attempting to sync node with API server"
Jul 06 18:17:45.602306 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.280666    2580 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 06 18:17:45.602306 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.280679    2580 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 06 18:17:45.602306 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.280691    2580 kubelet.go:398] "Adding apiserver pod source"
Jul 06 18:17:45.602306 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.280699    2580 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 06 18:17:45.602306 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.280988    2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 06 18:17:45.602869 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.281002    2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 06 18:17:45.602869 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.283744    2580 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1"
Jul 06 18:17:45.602869 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.284943    2580 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 06 18:17:45.602869 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.284971    2580 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 06 18:17:45.602869 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.286465    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 06 18:17:45.602869 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.286491    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 06 18:17:45.602869 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.286504    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 06 18:17:45.602869 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.286516    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 06 18:17:45.602869 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.286538    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 06 18:17:45.602869 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.286545    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 06 18:17:45.602869 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.286554    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 06 18:17:45.602869 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.286560    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 06 18:17:45.602869 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.286566    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 06 18:17:45.602869 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.286573    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 06 18:17:45.602869 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.286586    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 06 18:17:45.602869 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.286594    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 06 18:17:45.602869 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.287668    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 06 18:17:45.602869 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.287676    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 06 18:17:45.602869 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.290709    2580 server.go:1268] "Started kubelet"
Jul 06 18:17:45.602869 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.290806    2580 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 06 18:17:45.602869 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.290903    2580 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 06 18:17:45.602869 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.290988    2580 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 06 18:17:45.602869 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.292008    2580 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-134-113.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 06 18:17:45.602869 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:17:43.292009    2580 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-134-113.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 06 18:17:45.602869 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:17:43.292109    2580 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 06 18:17:45.602869 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.292219    2580 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 06 18:17:45.602869 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.293631    2580 server.go:310] "Adding debug handlers to kubelet server"
Jul 06 18:17:45.602869 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:17:43.296810    2580 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 06 18:17:45.602869 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:17:43.296248    2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-134-113.ec2.internal.18bfc62a7f6b79ab  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-113.ec2.internal,UID:ip-10-0-134-113.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-134-113.ec2.internal,},FirstTimestamp:2026-07-06 18:17:43.290681771 +0000 UTC m=+0.342726283,LastTimestamp:2026-07-06 18:17:43.290681771 +0000 UTC m=+0.342726283,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-113.ec2.internal,}"
Jul 06 18:17:45.602869 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.298339    2580 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 06 18:17:45.602869 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.298351    2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 06 18:17:45.602869 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:17:43.298470    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-113.ec2.internal\" not found"
Jul 06 18:17:45.602869 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.298947    2580 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 06 18:17:45.602869 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.298974    2580 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 06 18:17:45.602869 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.298987    2580 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 06 18:17:45.602869 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.299063    2580 reconstruct.go:97] "Volume reconstruction finished"
Jul 06 18:17:45.602869 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.299074    2580 reconciler.go:29] "Reconciler: start to sync state"
Jul 06 18:17:45.602869 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.299237    2580 factory.go:55] Registering systemd factory
Jul 06 18:17:45.602869 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.299248    2580 factory.go:223] Registration of the systemd container factory successfully
Jul 06 18:17:45.602869 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.299501    2580 factory.go:153] Registering CRI-O factory
Jul 06 18:17:45.602869 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.299511    2580 factory.go:223] Registration of the crio container factory successfully
Jul 06 18:17:45.801426 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.299548    2580 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 06 18:17:45.801426 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.299572    2580 factory.go:103] Registering Raw factory
Jul 06 18:17:45.801426 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.299585    2580 manager.go:1196] Started watching for new ooms in manager
Jul 06 18:17:45.801426 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.299966    2580 manager.go:349] Starting recovery of all containers
Jul 06 18:17:45.801426 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.306922    2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 06 18:17:45.801426 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:17:43.308866    2580 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 06 18:17:45.801426 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:17:43.308868    2580 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-134-113.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 06 18:17:45.801426 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.312546    2580 manager.go:354] Recovery completed
Jul 06 18:17:45.801426 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.318142    2580 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-rbbtb"
Jul 06 18:17:45.801426 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.320415    2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-rbbtb"
Jul 06 18:17:45.801426 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.323308    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 06 18:17:45.801426 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.325202    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-113.ec2.internal" event="NodeHasSufficientMemory"
Jul 06 18:17:45.801426 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.325222    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-113.ec2.internal" event="NodeHasNoDiskPressure"
Jul 06 18:17:45.801426 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.325232    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-113.ec2.internal" event="NodeHasSufficientPID"
Jul 06 18:17:45.801426 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.325825    2580 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 06 18:17:45.801426 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.325835    2580 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 06 18:17:45.801426 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.325848    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 06 18:17:45.801426 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.325850    2580 state_mem.go:36] "Initialized new in-memory state store"
Jul 06 18:17:45.801426 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.326596    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-113.ec2.internal" event="NodeHasSufficientMemory"
Jul 06 18:17:45.801426 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.326629    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-113.ec2.internal" event="NodeHasNoDiskPressure"
Jul 06 18:17:45.801426 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.326638    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-113.ec2.internal" event="NodeHasSufficientPID"
Jul 06 18:17:45.801426 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.327099    2580 policy_none.go:49] "None policy: Start"
Jul 06 18:17:45.801426 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.327116    2580 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 06 18:17:45.801426 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.327129    2580 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 06 18:17:45.801426 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:17:43.327716    2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-134-113.ec2.internal.18bfc62a817a60c6  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-113.ec2.internal,UID:ip-10-0-134-113.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-134-113.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-134-113.ec2.internal,},FirstTimestamp:2026-07-06 18:17:43.32521287 +0000 UTC m=+0.377257385,LastTimestamp:2026-07-06 18:17:43.32521287 +0000 UTC m=+0.377257385,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-113.ec2.internal,}"
Jul 06 18:17:45.801426 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.329049    2580 policy_none.go:47] "Start"
Jul 06 18:17:45.801426 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.363963    2580 manager.go:337] "Starting Device Plugin manager"
Jul 06 18:17:45.801426 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:17:43.364157    2580 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 06 18:17:45.801426 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.364172    2580 server.go:87] "Starting device plugin registration server"
Jul 06 18:17:45.801426 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.364921    2580 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 06 18:17:45.801426 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.364936    2580 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 06 18:17:45.801426 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.365049    2580 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 06 18:17:45.801426 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.365143    2580 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 06 18:17:45.801426 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.365153    2580 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 06 18:17:45.801426 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:17:43.365650    2580 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 06 18:17:45.801426 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:17:43.365697    2580 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-134-113.ec2.internal\" not found"
Jul 06 18:17:45.801426 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.421924    2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 06 18:17:45.801426 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.421965    2580 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 06 18:17:45.801426 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.421988    2580 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 06 18:17:45.803259 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:17:43.422020    2580 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 06 18:17:45.803259 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:43.424661    2580 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 06 18:17:45.803259 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:45.360314    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 06 18:17:45.803259 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:45.363939    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-113.ec2.internal" event="NodeHasSufficientMemory"
Jul 06 18:17:45.803259 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:45.363968    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-113.ec2.internal" event="NodeHasNoDiskPressure"
Jul 06 18:17:45.803259 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:45.363983    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-113.ec2.internal" event="NodeHasSufficientPID"
Jul 06 18:17:45.803259 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:45.364006    2580 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-134-113.ec2.internal"
Jul 06 18:17:45.803259 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:45.371798    2580 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-134-113.ec2.internal"
Jul 06 18:17:45.803259 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:17:45.371819    2580 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-134-113.ec2.internal\": node \"ip-10-0-134-113.ec2.internal\" not found"
Jul 06 18:17:45.803259 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:17:45.381144    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-113.ec2.internal\" not found"
Jul 06 18:17:45.803259 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:45.417121    2580 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-113.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-134-113.ec2.internal"]
Jul 06 18:17:45.803259 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:45.417196    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 06 18:17:45.803259 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:45.418698    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-113.ec2.internal" event="NodeHasSufficientMemory"
Jul 06 18:17:45.803259 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:45.418723    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-113.ec2.internal" event="NodeHasNoDiskPressure"
Jul 06 18:17:45.803259 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:45.418733    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-113.ec2.internal" event="NodeHasSufficientPID"
Jul 06 18:17:45.803259 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:45.419757    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 06 18:17:45.803259 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:45.419866    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-113.ec2.internal"
Jul 06 18:17:45.803259 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:45.419895    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 06 18:17:45.803259 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:45.420437    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-113.ec2.internal" event="NodeHasSufficientMemory"
Jul 06 18:17:45.803259 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:45.420462    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-113.ec2.internal" event="NodeHasNoDiskPressure"
Jul 06 18:17:45.803259 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:45.420472    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-113.ec2.internal" event="NodeHasSufficientPID"
Jul 06 18:17:45.803259 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:45.420430    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-113.ec2.internal" event="NodeHasSufficientMemory"
Jul 06 18:17:45.803259 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:45.420539    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-113.ec2.internal" event="NodeHasNoDiskPressure"
Jul 06 18:17:45.803259 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:45.420551    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-113.ec2.internal" event="NodeHasSufficientPID"
Jul 06 18:17:45.803259 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:45.422330    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-113.ec2.internal"
Jul 06 18:17:45.803259 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:45.422356    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 06 18:17:45.803259 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:45.423001    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-113.ec2.internal" event="NodeHasSufficientMemory"
Jul 06 18:17:45.803259 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:45.423037    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-113.ec2.internal" event="NodeHasNoDiskPressure"
Jul 06 18:17:45.803259 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:45.423048    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-113.ec2.internal" event="NodeHasSufficientPID"
Jul 06 18:17:45.803259 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:17:45.440502    2580 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-134-113.ec2.internal\" not found" node="ip-10-0-134-113.ec2.internal"
Jul 06 18:17:45.803259 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:17:45.445798    2580 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-134-113.ec2.internal\" not found" node="ip-10-0-134-113.ec2.internal"
Jul 06 18:17:45.803259 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:17:45.482050    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-113.ec2.internal\" not found"
Jul 06 18:17:45.803259 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:45.495341    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/6eab86221f4cc923c73e223abc50211e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-113.ec2.internal\" (UID: \"6eab86221f4cc923c73e223abc50211e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-113.ec2.internal"
Jul 06 18:17:45.803259 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:45.495364    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6eab86221f4cc923c73e223abc50211e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-113.ec2.internal\" (UID: \"6eab86221f4cc923c73e223abc50211e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-113.ec2.internal"
Jul 06 18:17:45.804190 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:45.495381    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/66d40220305fe03479bbe46175f2cd79-config\") pod \"kube-apiserver-proxy-ip-10-0-134-113.ec2.internal\" (UID: \"66d40220305fe03479bbe46175f2cd79\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-113.ec2.internal"
Jul 06 18:17:45.804190 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:17:45.582699    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-113.ec2.internal\" not found"
Jul 06 18:17:45.804190 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:45.596031    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/6eab86221f4cc923c73e223abc50211e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-113.ec2.internal\" (UID: \"6eab86221f4cc923c73e223abc50211e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-113.ec2.internal"
Jul 06 18:17:45.804190 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:45.596056    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6eab86221f4cc923c73e223abc50211e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-113.ec2.internal\" (UID: \"6eab86221f4cc923c73e223abc50211e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-113.ec2.internal"
Jul 06 18:17:45.804190 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:45.596079    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/66d40220305fe03479bbe46175f2cd79-config\") pod \"kube-apiserver-proxy-ip-10-0-134-113.ec2.internal\" (UID: \"66d40220305fe03479bbe46175f2cd79\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-113.ec2.internal"
Jul 06 18:17:45.804190 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:45.596116    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/66d40220305fe03479bbe46175f2cd79-config\") pod \"kube-apiserver-proxy-ip-10-0-134-113.ec2.internal\" (UID: \"66d40220305fe03479bbe46175f2cd79\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-113.ec2.internal"
Jul 06 18:17:45.804190 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:45.596119    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/6eab86221f4cc923c73e223abc50211e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-113.ec2.internal\" (UID: \"6eab86221f4cc923c73e223abc50211e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-113.ec2.internal"
Jul 06 18:17:45.804190 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:45.596126    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6eab86221f4cc923c73e223abc50211e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-113.ec2.internal\" (UID: \"6eab86221f4cc923c73e223abc50211e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-113.ec2.internal"
Jul 06 18:17:45.804190 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:17:45.683489    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-113.ec2.internal\" not found"
Jul 06 18:17:45.804190 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:45.743732    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-113.ec2.internal"
Jul 06 18:17:45.804190 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:45.748544    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-113.ec2.internal"
Jul 06 18:17:45.804190 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:17:45.783932    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-113.ec2.internal\" not found"
Jul 06 18:17:46.024690 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:17:45.884402    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-113.ec2.internal\" not found"
Jul 06 18:17:46.024690 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:17:45.984936    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-113.ec2.internal\" not found"
Jul 06 18:17:46.238244 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.046950    2580 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 06 18:17:46.238244 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.094059    2580 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-113.ec2.internal"
Jul 06 18:17:46.238244 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.105449    2580 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 06 18:17:46.238244 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.107035    2580 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-113.ec2.internal"
Jul 06 18:17:46.238244 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.116275    2580 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 06 18:17:46.238244 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.135343    2580 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 06 18:17:46.238244 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.135458    2580 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 06 18:17:46.238244 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.135474    2580 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 06 18:17:46.238244 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.175577    2580 apiserver.go:52] "Watching apiserver"
Jul 06 18:17:46.238244 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.192157    2580 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 06 18:17:46.238244 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.192474    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/konnectivity-agent-b7jm9","openshift-dns/node-resolver-x7rd5","openshift-multus/multus-additional-cni-plugins-sl8sj","openshift-multus/multus-zk2x7","openshift-multus/network-metrics-daemon-xjbzr","openshift-network-operator/iptables-alerter-vz7bl","openshift-ovn-kubernetes/ovnkube-node-9h479","kube-system/kube-apiserver-proxy-ip-10-0-134-113.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ztxm5","openshift-cluster-node-tuning-operator/tuned-rttvb","openshift-image-registry/node-ca-kcr4l","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-113.ec2.internal","openshift-network-diagnostics/network-check-target-fp8fn","kube-system/global-pull-secret-syncer-xzqch"]
Jul 06 18:17:46.238244 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.193144    2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 06 18:17:46.238244 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.193221    2580 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 06 18:17:46.238244 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.193807    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-9h479"
Jul 06 18:17:46.238244 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.194827    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fp8fn"
Jul 06 18:17:46.238244 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:17:46.194879    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fp8fn" podUID="4e2f69c5-ef58-430a-8b74-eb53ce230c7e"
Jul 06 18:17:46.238244 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.196059    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-sl8sj"
Jul 06 18:17:46.238244 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.196390    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-vz5jb\""
Jul 06 18:17:46.238244 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.196587    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 06 18:17:46.238244 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.196608    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 06 18:17:46.238244 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.196674    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 06 18:17:46.238244 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.196687    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 06 18:17:46.238244 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.196834    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 06 18:17:46.238244 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.197201    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-zk2x7"
Jul 06 18:17:46.238244 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.197950    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 06 18:17:46.238244 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.198324    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xjbzr"
Jul 06 18:17:46.238244 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:17:46.198374    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xjbzr" podUID="f22387dc-2c30-4f80-8a80-3f6420c1c858"
Jul 06 18:17:46.238244 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.199074    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 06 18:17:46.238244 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.199161    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/5cfed2e4-8993-42f8-bc84-f9399b287729-ovnkube-script-lib\") pod \"ovnkube-node-9h479\" (UID: \"5cfed2e4-8993-42f8-bc84-f9399b287729\") " pod="openshift-ovn-kubernetes/ovnkube-node-9h479"
Jul 06 18:17:46.239909 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.199188    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5cfed2e4-8993-42f8-bc84-f9399b287729-host-slash\") pod \"ovnkube-node-9h479\" (UID: \"5cfed2e4-8993-42f8-bc84-f9399b287729\") " pod="openshift-ovn-kubernetes/ovnkube-node-9h479"
Jul 06 18:17:46.239909 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.199214    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5cfed2e4-8993-42f8-bc84-f9399b287729-etc-openvswitch\") pod \"ovnkube-node-9h479\" (UID: \"5cfed2e4-8993-42f8-bc84-f9399b287729\") " pod="openshift-ovn-kubernetes/ovnkube-node-9h479"
Jul 06 18:17:46.239909 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.199237    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5cfed2e4-8993-42f8-bc84-f9399b287729-run-openvswitch\") pod \"ovnkube-node-9h479\" (UID: \"5cfed2e4-8993-42f8-bc84-f9399b287729\") " pod="openshift-ovn-kubernetes/ovnkube-node-9h479"
Jul 06 18:17:46.239909 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.199262    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/5cfed2e4-8993-42f8-bc84-f9399b287729-log-socket\") pod \"ovnkube-node-9h479\" (UID: \"5cfed2e4-8993-42f8-bc84-f9399b287729\") " pod="openshift-ovn-kubernetes/ovnkube-node-9h479"
Jul 06 18:17:46.239909 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.199285    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5cfed2e4-8993-42f8-bc84-f9399b287729-host-run-ovn-kubernetes\") pod \"ovnkube-node-9h479\" (UID: \"5cfed2e4-8993-42f8-bc84-f9399b287729\") " pod="openshift-ovn-kubernetes/ovnkube-node-9h479"
Jul 06 18:17:46.239909 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.199307    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5cfed2e4-8993-42f8-bc84-f9399b287729-host-cni-bin\") pod \"ovnkube-node-9h479\" (UID: \"5cfed2e4-8993-42f8-bc84-f9399b287729\") " pod="openshift-ovn-kubernetes/ovnkube-node-9h479"
Jul 06 18:17:46.239909 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.199375    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/5cfed2e4-8993-42f8-bc84-f9399b287729-host-kubelet\") pod \"ovnkube-node-9h479\" (UID: \"5cfed2e4-8993-42f8-bc84-f9399b287729\") " pod="openshift-ovn-kubernetes/ovnkube-node-9h479"
Jul 06 18:17:46.239909 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.199395    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-vz7bl"
Jul 06 18:17:46.239909 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.199412    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/5cfed2e4-8993-42f8-bc84-f9399b287729-systemd-units\") pod \"ovnkube-node-9h479\" (UID: \"5cfed2e4-8993-42f8-bc84-f9399b287729\") " pod="openshift-ovn-kubernetes/ovnkube-node-9h479"
Jul 06 18:17:46.239909 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.199435    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/5cfed2e4-8993-42f8-bc84-f9399b287729-run-systemd\") pod \"ovnkube-node-9h479\" (UID: \"5cfed2e4-8993-42f8-bc84-f9399b287729\") " pod="openshift-ovn-kubernetes/ovnkube-node-9h479"
Jul 06 18:17:46.239909 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.199458    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/5cfed2e4-8993-42f8-bc84-f9399b287729-env-overrides\") pod \"ovnkube-node-9h479\" (UID: \"5cfed2e4-8993-42f8-bc84-f9399b287729\") " pod="openshift-ovn-kubernetes/ovnkube-node-9h479"
Jul 06 18:17:46.239909 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.199476    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 06 18:17:46.239909 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.199500    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5cfed2e4-8993-42f8-bc84-f9399b287729-var-lib-openvswitch\") pod \"ovnkube-node-9h479\" (UID: \"5cfed2e4-8993-42f8-bc84-f9399b287729\") " pod="openshift-ovn-kubernetes/ovnkube-node-9h479"
Jul 06 18:17:46.239909 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.199533    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/5cfed2e4-8993-42f8-bc84-f9399b287729-ovnkube-config\") pod \"ovnkube-node-9h479\" (UID: \"5cfed2e4-8993-42f8-bc84-f9399b287729\") " pod="openshift-ovn-kubernetes/ovnkube-node-9h479"
Jul 06 18:17:46.239909 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.199559    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/5cfed2e4-8993-42f8-bc84-f9399b287729-ovn-node-metrics-cert\") pod \"ovnkube-node-9h479\" (UID: \"5cfed2e4-8993-42f8-bc84-f9399b287729\") " pod="openshift-ovn-kubernetes/ovnkube-node-9h479"
Jul 06 18:17:46.239909 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.199582    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v5s29\" (UniqueName: \"kubernetes.io/projected/5cfed2e4-8993-42f8-bc84-f9399b287729-kube-api-access-v5s29\") pod \"ovnkube-node-9h479\" (UID: \"5cfed2e4-8993-42f8-bc84-f9399b287729\") " pod="openshift-ovn-kubernetes/ovnkube-node-9h479"
Jul 06 18:17:46.239909 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.199609    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-knzwx\" (UniqueName: \"kubernetes.io/projected/4e2f69c5-ef58-430a-8b74-eb53ce230c7e-kube-api-access-knzwx\") pod \"network-check-target-fp8fn\" (UID: \"4e2f69c5-ef58-430a-8b74-eb53ce230c7e\") " pod="openshift-network-diagnostics/network-check-target-fp8fn"
Jul 06 18:17:46.240773 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.199632    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5cfed2e4-8993-42f8-bc84-f9399b287729-host-run-netns\") pod \"ovnkube-node-9h479\" (UID: \"5cfed2e4-8993-42f8-bc84-f9399b287729\") " pod="openshift-ovn-kubernetes/ovnkube-node-9h479"
Jul 06 18:17:46.240773 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.199654    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/5cfed2e4-8993-42f8-bc84-f9399b287729-run-ovn\") pod \"ovnkube-node-9h479\" (UID: \"5cfed2e4-8993-42f8-bc84-f9399b287729\") " pod="openshift-ovn-kubernetes/ovnkube-node-9h479"
Jul 06 18:17:46.240773 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.199676    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/5cfed2e4-8993-42f8-bc84-f9399b287729-node-log\") pod \"ovnkube-node-9h479\" (UID: \"5cfed2e4-8993-42f8-bc84-f9399b287729\") " pod="openshift-ovn-kubernetes/ovnkube-node-9h479"
Jul 06 18:17:46.240773 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.199703    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/5cfed2e4-8993-42f8-bc84-f9399b287729-host-cni-netd\") pod \"ovnkube-node-9h479\" (UID: \"5cfed2e4-8993-42f8-bc84-f9399b287729\") " pod="openshift-ovn-kubernetes/ovnkube-node-9h479"
Jul 06 18:17:46.240773 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.199729    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5cfed2e4-8993-42f8-bc84-f9399b287729-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-9h479\" (UID: \"5cfed2e4-8993-42f8-bc84-f9399b287729\") " pod="openshift-ovn-kubernetes/ovnkube-node-9h479"
Jul 06 18:17:46.240773 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.199758    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 06 18:17:46.240773 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.199781    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 06 18:17:46.240773 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.199831    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 06 18:17:46.240773 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.199900    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-tfsbl\""
Jul 06 18:17:46.240773 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.199900    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 06 18:17:46.240773 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.200439    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-fsp4r\""
Jul 06 18:17:46.240773 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.200634    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ztxm5"
Jul 06 18:17:46.240773 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.202202    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-qp82k\""
Jul 06 18:17:46.240773 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.202437    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 06 18:17:46.240773 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.202586    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 06 18:17:46.240773 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.205063    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 06 18:17:46.240773 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.205120    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-b7jm9"
Jul 06 18:17:46.240773 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.206073    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 06 18:17:46.240773 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.206139    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-mtfz8\""
Jul 06 18:17:46.240773 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.206408    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 06 18:17:46.240773 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.206419    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 06 18:17:46.240773 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.207209    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-x7rd5"
Jul 06 18:17:46.240773 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.207784    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 06 18:17:46.240773 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.207819    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-jwdcf\""
Jul 06 18:17:46.240773 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.207914    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 06 18:17:46.240773 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.208457    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-rttvb"
Jul 06 18:17:46.240773 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.208608    2580 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 06 18:17:46.240773 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.209799    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-kcr4l"
Jul 06 18:17:46.240773 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.209843    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 06 18:17:46.240773 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.209886    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-cmhhj\""
Jul 06 18:17:46.240773 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.210131    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 06 18:17:46.241862 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.210740    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 06 18:17:46.241862 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.210818    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-mvlrl\""
Jul 06 18:17:46.241862 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.210819    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 06 18:17:46.241862 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.211062    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xzqch"
Jul 06 18:17:46.241862 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:17:46.211108    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xzqch" podUID="3b6b323e-eca7-4c68-87a7-0d48a9114a2c"
Jul 06 18:17:46.241862 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.212531    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-x8b2h\""
Jul 06 18:17:46.241862 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.212537    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 06 18:17:46.241862 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.212532    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 06 18:17:46.241862 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.212785    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 06 18:17:46.241862 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.217046    2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-05 18:12:45 +0000 UTC" deadline="2028-04-01 19:43:19.032567492 +0000 UTC"
Jul 06 18:17:46.241862 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.217076    2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15241h25m32.815493588s"
Jul 06 18:17:46.241862 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.225570    2580 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-7jppj"
Jul 06 18:17:46.241862 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.233272    2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-7jppj"
Jul 06 18:17:46.403082 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.249764    2580 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 06 18:17:46.403082 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:46.274751    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6eab86221f4cc923c73e223abc50211e.slice/crio-941babc7f25a3a02d53fc52f969f9ecb0115419e07a43b6bafc4e1356834cdd7 WatchSource:0}: Error finding container 941babc7f25a3a02d53fc52f969f9ecb0115419e07a43b6bafc4e1356834cdd7: Status 404 returned error can't find the container with id 941babc7f25a3a02d53fc52f969f9ecb0115419e07a43b6bafc4e1356834cdd7
Jul 06 18:17:46.403082 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:46.275454    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66d40220305fe03479bbe46175f2cd79.slice/crio-3e1411ce36a4936f930024a60eca4be30d939585778258bd8f30807681b87723 WatchSource:0}: Error finding container 3e1411ce36a4936f930024a60eca4be30d939585778258bd8f30807681b87723: Status 404 returned error can't find the container with id 3e1411ce36a4936f930024a60eca4be30d939585778258bd8f30807681b87723
Jul 06 18:17:46.403082 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.286397    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 06 18:17:46.403082 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.294526    2580 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 06 18:17:46.403082 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.300804    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/0474b033-5051-4df3-8135-4b05741835ac-registration-dir\") pod \"aws-ebs-csi-driver-node-ztxm5\" (UID: \"0474b033-5051-4df3-8135-4b05741835ac\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ztxm5"
Jul 06 18:17:46.403082 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.300825    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/0474b033-5051-4df3-8135-4b05741835ac-etc-selinux\") pod \"aws-ebs-csi-driver-node-ztxm5\" (UID: \"0474b033-5051-4df3-8135-4b05741835ac\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ztxm5"
Jul 06 18:17:46.403082 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.300843    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a420108c-26d7-46d2-a3d5-3b354aa01af0-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-sl8sj\" (UID: \"a420108c-26d7-46d2-a3d5-3b354aa01af0\") " pod="openshift-multus/multus-additional-cni-plugins-sl8sj"
Jul 06 18:17:46.403082 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.300859    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/7b707150-01d0-47c3-b61e-a9a6f33465fc-etc-modprobe-d\") pod \"tuned-rttvb\" (UID: \"7b707150-01d0-47c3-b61e-a9a6f33465fc\") " pod="openshift-cluster-node-tuning-operator/tuned-rttvb"
Jul 06 18:17:46.403082 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.300873    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c5411d9d-cd87-44a4-a2b8-99acc89e5a2f-etc-kubernetes\") pod \"multus-zk2x7\" (UID: \"c5411d9d-cd87-44a4-a2b8-99acc89e5a2f\") " pod="openshift-multus/multus-zk2x7"
Jul 06 18:17:46.403082 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.300886    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/af914b85-02d3-423a-aed5-e96b26e6c1c1-iptables-alerter-script\") pod \"iptables-alerter-vz7bl\" (UID: \"af914b85-02d3-423a-aed5-e96b26e6c1c1\") " pod="openshift-network-operator/iptables-alerter-vz7bl"
Jul 06 18:17:46.403082 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.300916    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l4rrx\" (UniqueName: \"kubernetes.io/projected/0474b033-5051-4df3-8135-4b05741835ac-kube-api-access-l4rrx\") pod \"aws-ebs-csi-driver-node-ztxm5\" (UID: \"0474b033-5051-4df3-8135-4b05741835ac\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ztxm5"
Jul 06 18:17:46.403082 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.300930    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a420108c-26d7-46d2-a3d5-3b354aa01af0-system-cni-dir\") pod \"multus-additional-cni-plugins-sl8sj\" (UID: \"a420108c-26d7-46d2-a3d5-3b354aa01af0\") " pod="openshift-multus/multus-additional-cni-plugins-sl8sj"
Jul 06 18:17:46.403082 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.300947    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/5cfed2e4-8993-42f8-bc84-f9399b287729-run-ovn\") pod \"ovnkube-node-9h479\" (UID: \"5cfed2e4-8993-42f8-bc84-f9399b287729\") " pod="openshift-ovn-kubernetes/ovnkube-node-9h479"
Jul 06 18:17:46.403082 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.300961    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a420108c-26d7-46d2-a3d5-3b354aa01af0-cni-binary-copy\") pod \"multus-additional-cni-plugins-sl8sj\" (UID: \"a420108c-26d7-46d2-a3d5-3b354aa01af0\") " pod="openshift-multus/multus-additional-cni-plugins-sl8sj"
Jul 06 18:17:46.403082 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.300997    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5cfed2e4-8993-42f8-bc84-f9399b287729-host-slash\") pod \"ovnkube-node-9h479\" (UID: \"5cfed2e4-8993-42f8-bc84-f9399b287729\") " pod="openshift-ovn-kubernetes/ovnkube-node-9h479"
Jul 06 18:17:46.403082 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.301025    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5cfed2e4-8993-42f8-bc84-f9399b287729-run-openvswitch\") pod \"ovnkube-node-9h479\" (UID: \"5cfed2e4-8993-42f8-bc84-f9399b287729\") " pod="openshift-ovn-kubernetes/ovnkube-node-9h479"
Jul 06 18:17:46.404699 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.301045    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5cfed2e4-8993-42f8-bc84-f9399b287729-host-run-ovn-kubernetes\") pod \"ovnkube-node-9h479\" (UID: \"5cfed2e4-8993-42f8-bc84-f9399b287729\") " pod="openshift-ovn-kubernetes/ovnkube-node-9h479"
Jul 06 18:17:46.404699 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.301058    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5cfed2e4-8993-42f8-bc84-f9399b287729-host-cni-bin\") pod \"ovnkube-node-9h479\" (UID: \"5cfed2e4-8993-42f8-bc84-f9399b287729\") " pod="openshift-ovn-kubernetes/ovnkube-node-9h479"
Jul 06 18:17:46.404699 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.301074    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/0474b033-5051-4df3-8135-4b05741835ac-sys-fs\") pod \"aws-ebs-csi-driver-node-ztxm5\" (UID: \"0474b033-5051-4df3-8135-4b05741835ac\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ztxm5"
Jul 06 18:17:46.404699 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.301089    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f117fdd5-8a61-484e-8ebc-668f89723813-konnectivity-ca\") pod \"konnectivity-agent-b7jm9\" (UID: \"f117fdd5-8a61-484e-8ebc-668f89723813\") " pod="kube-system/konnectivity-agent-b7jm9"
Jul 06 18:17:46.404699 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.301103    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7b707150-01d0-47c3-b61e-a9a6f33465fc-host\") pod \"tuned-rttvb\" (UID: \"7b707150-01d0-47c3-b61e-a9a6f33465fc\") " pod="openshift-cluster-node-tuning-operator/tuned-rttvb"
Jul 06 18:17:46.404699 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.301127    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fdvzt\" (UniqueName: \"kubernetes.io/projected/7b707150-01d0-47c3-b61e-a9a6f33465fc-kube-api-access-fdvzt\") pod \"tuned-rttvb\" (UID: \"7b707150-01d0-47c3-b61e-a9a6f33465fc\") " pod="openshift-cluster-node-tuning-operator/tuned-rttvb"
Jul 06 18:17:46.404699 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.301141    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c5411d9d-cd87-44a4-a2b8-99acc89e5a2f-cnibin\") pod \"multus-zk2x7\" (UID: \"c5411d9d-cd87-44a4-a2b8-99acc89e5a2f\") " pod="openshift-multus/multus-zk2x7"
Jul 06 18:17:46.404699 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.301154    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/c5411d9d-cd87-44a4-a2b8-99acc89e5a2f-multus-socket-dir-parent\") pod \"multus-zk2x7\" (UID: \"c5411d9d-cd87-44a4-a2b8-99acc89e5a2f\") " pod="openshift-multus/multus-zk2x7"
Jul 06 18:17:46.404699 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.301176    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/c5411d9d-cd87-44a4-a2b8-99acc89e5a2f-host-var-lib-cni-multus\") pod \"multus-zk2x7\" (UID: \"c5411d9d-cd87-44a4-a2b8-99acc89e5a2f\") " pod="openshift-multus/multus-zk2x7"
Jul 06 18:17:46.404699 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.301191    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c5411d9d-cd87-44a4-a2b8-99acc89e5a2f-host-var-lib-kubelet\") pod \"multus-zk2x7\" (UID: \"c5411d9d-cd87-44a4-a2b8-99acc89e5a2f\") " pod="openshift-multus/multus-zk2x7"
Jul 06 18:17:46.404699 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.301204    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kf4f6\" (UniqueName: \"kubernetes.io/projected/c5411d9d-cd87-44a4-a2b8-99acc89e5a2f-kube-api-access-kf4f6\") pod \"multus-zk2x7\" (UID: \"c5411d9d-cd87-44a4-a2b8-99acc89e5a2f\") " pod="openshift-multus/multus-zk2x7"
Jul 06 18:17:46.404699 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.301230    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f117fdd5-8a61-484e-8ebc-668f89723813-agent-certs\") pod \"konnectivity-agent-b7jm9\" (UID: \"f117fdd5-8a61-484e-8ebc-668f89723813\") " pod="kube-system/konnectivity-agent-b7jm9"
Jul 06 18:17:46.404699 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.301246    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/5cfed2e4-8993-42f8-bc84-f9399b287729-ovnkube-config\") pod \"ovnkube-node-9h479\" (UID: \"5cfed2e4-8993-42f8-bc84-f9399b287729\") " pod="openshift-ovn-kubernetes/ovnkube-node-9h479"
Jul 06 18:17:46.404699 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.301259    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d27a6ce1-0910-4c08-959a-550b8d468219-tmp-dir\") pod \"node-resolver-x7rd5\" (UID: \"d27a6ce1-0910-4c08-959a-550b8d468219\") " pod="openshift-dns/node-resolver-x7rd5"
Jul 06 18:17:46.404699 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.301272    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/7b707150-01d0-47c3-b61e-a9a6f33465fc-etc-sysconfig\") pod \"tuned-rttvb\" (UID: \"7b707150-01d0-47c3-b61e-a9a6f33465fc\") " pod="openshift-cluster-node-tuning-operator/tuned-rttvb"
Jul 06 18:17:46.404699 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.301289    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7b707150-01d0-47c3-b61e-a9a6f33465fc-sys\") pod \"tuned-rttvb\" (UID: \"7b707150-01d0-47c3-b61e-a9a6f33465fc\") " pod="openshift-cluster-node-tuning-operator/tuned-rttvb"
Jul 06 18:17:46.404699 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.301316    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c5411d9d-cd87-44a4-a2b8-99acc89e5a2f-cni-binary-copy\") pod \"multus-zk2x7\" (UID: \"c5411d9d-cd87-44a4-a2b8-99acc89e5a2f\") " pod="openshift-multus/multus-zk2x7"
Jul 06 18:17:46.406442 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.301328    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/af914b85-02d3-423a-aed5-e96b26e6c1c1-host-slash\") pod \"iptables-alerter-vz7bl\" (UID: \"af914b85-02d3-423a-aed5-e96b26e6c1c1\") " pod="openshift-network-operator/iptables-alerter-vz7bl"
Jul 06 18:17:46.406442 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.301344    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/5cfed2e4-8993-42f8-bc84-f9399b287729-ovnkube-script-lib\") pod \"ovnkube-node-9h479\" (UID: \"5cfed2e4-8993-42f8-bc84-f9399b287729\") " pod="openshift-ovn-kubernetes/ovnkube-node-9h479"
Jul 06 18:17:46.406442 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.301357    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/0474b033-5051-4df3-8135-4b05741835ac-socket-dir\") pod \"aws-ebs-csi-driver-node-ztxm5\" (UID: \"0474b033-5051-4df3-8135-4b05741835ac\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ztxm5"
Jul 06 18:17:46.406442 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.301371    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a420108c-26d7-46d2-a3d5-3b354aa01af0-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-sl8sj\" (UID: \"a420108c-26d7-46d2-a3d5-3b354aa01af0\") " pod="openshift-multus/multus-additional-cni-plugins-sl8sj"
Jul 06 18:17:46.406442 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.301391    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/5cfed2e4-8993-42f8-bc84-f9399b287729-log-socket\") pod \"ovnkube-node-9h479\" (UID: \"5cfed2e4-8993-42f8-bc84-f9399b287729\") " pod="openshift-ovn-kubernetes/ovnkube-node-9h479"
Jul 06 18:17:46.406442 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.301406    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/0474b033-5051-4df3-8135-4b05741835ac-device-dir\") pod \"aws-ebs-csi-driver-node-ztxm5\" (UID: \"0474b033-5051-4df3-8135-4b05741835ac\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ztxm5"
Jul 06 18:17:46.406442 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.301419    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/5cfed2e4-8993-42f8-bc84-f9399b287729-host-kubelet\") pod \"ovnkube-node-9h479\" (UID: \"5cfed2e4-8993-42f8-bc84-f9399b287729\") " pod="openshift-ovn-kubernetes/ovnkube-node-9h479"
Jul 06 18:17:46.406442 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.301434    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/5cfed2e4-8993-42f8-bc84-f9399b287729-run-systemd\") pod \"ovnkube-node-9h479\" (UID: \"5cfed2e4-8993-42f8-bc84-f9399b287729\") " pod="openshift-ovn-kubernetes/ovnkube-node-9h479"
Jul 06 18:17:46.406442 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.301458    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/5cfed2e4-8993-42f8-bc84-f9399b287729-env-overrides\") pod \"ovnkube-node-9h479\" (UID: \"5cfed2e4-8993-42f8-bc84-f9399b287729\") " pod="openshift-ovn-kubernetes/ovnkube-node-9h479"
Jul 06 18:17:46.406442 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.301471    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-56m7v\" (UniqueName: \"kubernetes.io/projected/f22387dc-2c30-4f80-8a80-3f6420c1c858-kube-api-access-56m7v\") pod \"network-metrics-daemon-xjbzr\" (UID: \"f22387dc-2c30-4f80-8a80-3f6420c1c858\") " pod="openshift-multus/network-metrics-daemon-xjbzr"
Jul 06 18:17:46.406442 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.301485    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7b707150-01d0-47c3-b61e-a9a6f33465fc-lib-modules\") pod \"tuned-rttvb\" (UID: \"7b707150-01d0-47c3-b61e-a9a6f33465fc\") " pod="openshift-cluster-node-tuning-operator/tuned-rttvb"
Jul 06 18:17:46.406442 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.301502    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/3b6b323e-eca7-4c68-87a7-0d48a9114a2c-dbus\") pod \"global-pull-secret-syncer-xzqch\" (UID: \"3b6b323e-eca7-4c68-87a7-0d48a9114a2c\") " pod="kube-system/global-pull-secret-syncer-xzqch"
Jul 06 18:17:46.406442 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.301529    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5cfed2e4-8993-42f8-bc84-f9399b287729-var-lib-openvswitch\") pod \"ovnkube-node-9h479\" (UID: \"5cfed2e4-8993-42f8-bc84-f9399b287729\") " pod="openshift-ovn-kubernetes/ovnkube-node-9h479"
Jul 06 18:17:46.406442 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.301551    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/5cfed2e4-8993-42f8-bc84-f9399b287729-ovn-node-metrics-cert\") pod \"ovnkube-node-9h479\" (UID: \"5cfed2e4-8993-42f8-bc84-f9399b287729\") " pod="openshift-ovn-kubernetes/ovnkube-node-9h479"
Jul 06 18:17:46.406442 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.301568    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-knzwx\" (UniqueName: \"kubernetes.io/projected/4e2f69c5-ef58-430a-8b74-eb53ce230c7e-kube-api-access-knzwx\") pod \"network-check-target-fp8fn\" (UID: \"4e2f69c5-ef58-430a-8b74-eb53ce230c7e\") " pod="openshift-network-diagnostics/network-check-target-fp8fn"
Jul 06 18:17:46.406442 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.301582    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7b707150-01d0-47c3-b61e-a9a6f33465fc-var-lib-kubelet\") pod \"tuned-rttvb\" (UID: \"7b707150-01d0-47c3-b61e-a9a6f33465fc\") " pod="openshift-cluster-node-tuning-operator/tuned-rttvb"
Jul 06 18:17:46.408152 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.301595    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/5cfed2e4-8993-42f8-bc84-f9399b287729-run-ovn\") pod \"ovnkube-node-9h479\" (UID: \"5cfed2e4-8993-42f8-bc84-f9399b287729\") " pod="openshift-ovn-kubernetes/ovnkube-node-9h479"
Jul 06 18:17:46.408152 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.301633    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/5cfed2e4-8993-42f8-bc84-f9399b287729-host-cni-netd\") pod \"ovnkube-node-9h479\" (UID: \"5cfed2e4-8993-42f8-bc84-f9399b287729\") " pod="openshift-ovn-kubernetes/ovnkube-node-9h479"
Jul 06 18:17:46.408152 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.301666    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5cfed2e4-8993-42f8-bc84-f9399b287729-host-slash\") pod \"ovnkube-node-9h479\" (UID: \"5cfed2e4-8993-42f8-bc84-f9399b287729\") " pod="openshift-ovn-kubernetes/ovnkube-node-9h479"
Jul 06 18:17:46.408152 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.301678    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/5cfed2e4-8993-42f8-bc84-f9399b287729-log-socket\") pod \"ovnkube-node-9h479\" (UID: \"5cfed2e4-8993-42f8-bc84-f9399b287729\") " pod="openshift-ovn-kubernetes/ovnkube-node-9h479"
Jul 06 18:17:46.408152 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.301694    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5cfed2e4-8993-42f8-bc84-f9399b287729-run-openvswitch\") pod \"ovnkube-node-9h479\" (UID: \"5cfed2e4-8993-42f8-bc84-f9399b287729\") " pod="openshift-ovn-kubernetes/ovnkube-node-9h479"
Jul 06 18:17:46.408152 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.301709    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/5cfed2e4-8993-42f8-bc84-f9399b287729-host-kubelet\") pod \"ovnkube-node-9h479\" (UID: \"5cfed2e4-8993-42f8-bc84-f9399b287729\") " pod="openshift-ovn-kubernetes/ovnkube-node-9h479"
Jul 06 18:17:46.408152 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.301723    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5cfed2e4-8993-42f8-bc84-f9399b287729-host-run-ovn-kubernetes\") pod \"ovnkube-node-9h479\" (UID: \"5cfed2e4-8993-42f8-bc84-f9399b287729\") " pod="openshift-ovn-kubernetes/ovnkube-node-9h479"
Jul 06 18:17:46.408152 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.301733    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/5cfed2e4-8993-42f8-bc84-f9399b287729-run-systemd\") pod \"ovnkube-node-9h479\" (UID: \"5cfed2e4-8993-42f8-bc84-f9399b287729\") " pod="openshift-ovn-kubernetes/ovnkube-node-9h479"
Jul 06 18:17:46.408152 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.301746    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5cfed2e4-8993-42f8-bc84-f9399b287729-host-cni-bin\") pod \"ovnkube-node-9h479\" (UID: \"5cfed2e4-8993-42f8-bc84-f9399b287729\") " pod="openshift-ovn-kubernetes/ovnkube-node-9h479"
Jul 06 18:17:46.408152 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.301596    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/5cfed2e4-8993-42f8-bc84-f9399b287729-host-cni-netd\") pod \"ovnkube-node-9h479\" (UID: \"5cfed2e4-8993-42f8-bc84-f9399b287729\") " pod="openshift-ovn-kubernetes/ovnkube-node-9h479"
Jul 06 18:17:46.408152 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.301838    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7b707150-01d0-47c3-b61e-a9a6f33465fc-etc-kubernetes\") pod \"tuned-rttvb\" (UID: \"7b707150-01d0-47c3-b61e-a9a6f33465fc\") " pod="openshift-cluster-node-tuning-operator/tuned-rttvb"
Jul 06 18:17:46.408152 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.301857    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7b707150-01d0-47c3-b61e-a9a6f33465fc-tmp\") pod \"tuned-rttvb\" (UID: \"7b707150-01d0-47c3-b61e-a9a6f33465fc\") " pod="openshift-cluster-node-tuning-operator/tuned-rttvb"
Jul 06 18:17:46.408152 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.301879    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c5411d9d-cd87-44a4-a2b8-99acc89e5a2f-os-release\") pod \"multus-zk2x7\" (UID: \"c5411d9d-cd87-44a4-a2b8-99acc89e5a2f\") " pod="openshift-multus/multus-zk2x7"
Jul 06 18:17:46.408152 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.301893    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/0474b033-5051-4df3-8135-4b05741835ac-kubelet-dir\") pod \"aws-ebs-csi-driver-node-ztxm5\" (UID: \"0474b033-5051-4df3-8135-4b05741835ac\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ztxm5"
Jul 06 18:17:46.408152 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.301908    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a420108c-26d7-46d2-a3d5-3b354aa01af0-cnibin\") pod \"multus-additional-cni-plugins-sl8sj\" (UID: \"a420108c-26d7-46d2-a3d5-3b354aa01af0\") " pod="openshift-multus/multus-additional-cni-plugins-sl8sj"
Jul 06 18:17:46.408152 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.301923    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a420108c-26d7-46d2-a3d5-3b354aa01af0-tuning-conf-dir\") pod \"multus-additional-cni-plugins-sl8sj\" (UID: \"a420108c-26d7-46d2-a3d5-3b354aa01af0\") " pod="openshift-multus/multus-additional-cni-plugins-sl8sj"
Jul 06 18:17:46.408152 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.301947    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5cfed2e4-8993-42f8-bc84-f9399b287729-etc-openvswitch\") pod \"ovnkube-node-9h479\" (UID: \"5cfed2e4-8993-42f8-bc84-f9399b287729\") " pod="openshift-ovn-kubernetes/ovnkube-node-9h479"
Jul 06 18:17:46.408796 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.301963    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/3b6b323e-eca7-4c68-87a7-0d48a9114a2c-kubelet-config\") pod \"global-pull-secret-syncer-xzqch\" (UID: \"3b6b323e-eca7-4c68-87a7-0d48a9114a2c\") " pod="kube-system/global-pull-secret-syncer-xzqch"
Jul 06 18:17:46.408796 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.301977    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c5411d9d-cd87-44a4-a2b8-99acc89e5a2f-multus-cni-dir\") pod \"multus-zk2x7\" (UID: \"c5411d9d-cd87-44a4-a2b8-99acc89e5a2f\") " pod="openshift-multus/multus-zk2x7"
Jul 06 18:17:46.408796 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.301992    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bbbnf\" (UniqueName: \"kubernetes.io/projected/a420108c-26d7-46d2-a3d5-3b354aa01af0-kube-api-access-bbbnf\") pod \"multus-additional-cni-plugins-sl8sj\" (UID: \"a420108c-26d7-46d2-a3d5-3b354aa01af0\") " pod="openshift-multus/multus-additional-cni-plugins-sl8sj"
Jul 06 18:17:46.408796 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.302029    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v5s29\" (UniqueName: \"kubernetes.io/projected/5cfed2e4-8993-42f8-bc84-f9399b287729-kube-api-access-v5s29\") pod \"ovnkube-node-9h479\" (UID: \"5cfed2e4-8993-42f8-bc84-f9399b287729\") " pod="openshift-ovn-kubernetes/ovnkube-node-9h479"
Jul 06 18:17:46.408796 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.302048    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/7b707150-01d0-47c3-b61e-a9a6f33465fc-etc-sysctl-conf\") pod \"tuned-rttvb\" (UID: \"7b707150-01d0-47c3-b61e-a9a6f33465fc\") " pod="openshift-cluster-node-tuning-operator/tuned-rttvb"
Jul 06 18:17:46.408796 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.302065    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/5cfed2e4-8993-42f8-bc84-f9399b287729-env-overrides\") pod \"ovnkube-node-9h479\" (UID: \"5cfed2e4-8993-42f8-bc84-f9399b287729\") " pod="openshift-ovn-kubernetes/ovnkube-node-9h479"
Jul 06 18:17:46.408796 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.302066    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c2ef9254-5ac3-45db-99a0-b5aff2a25d58-serviceca\") pod \"node-ca-kcr4l\" (UID: \"c2ef9254-5ac3-45db-99a0-b5aff2a25d58\") " pod="openshift-image-registry/node-ca-kcr4l"
Jul 06 18:17:46.408796 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.302102    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5cfed2e4-8993-42f8-bc84-f9399b287729-var-lib-openvswitch\") pod \"ovnkube-node-9h479\" (UID: \"5cfed2e4-8993-42f8-bc84-f9399b287729\") " pod="openshift-ovn-kubernetes/ovnkube-node-9h479"
Jul 06 18:17:46.408796 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.302118    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/c5411d9d-cd87-44a4-a2b8-99acc89e5a2f-hostroot\") pod \"multus-zk2x7\" (UID: \"c5411d9d-cd87-44a4-a2b8-99acc89e5a2f\") " pod="openshift-multus/multus-zk2x7"
Jul 06 18:17:46.408796 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.302135    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/c5411d9d-cd87-44a4-a2b8-99acc89e5a2f-host-run-multus-certs\") pod \"multus-zk2x7\" (UID: \"c5411d9d-cd87-44a4-a2b8-99acc89e5a2f\") " pod="openshift-multus/multus-zk2x7"
Jul 06 18:17:46.408796 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.302149    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jfnjw\" (UniqueName: \"kubernetes.io/projected/af914b85-02d3-423a-aed5-e96b26e6c1c1-kube-api-access-jfnjw\") pod \"iptables-alerter-vz7bl\" (UID: \"af914b85-02d3-423a-aed5-e96b26e6c1c1\") " pod="openshift-network-operator/iptables-alerter-vz7bl"
Jul 06 18:17:46.408796 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.302166    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a420108c-26d7-46d2-a3d5-3b354aa01af0-os-release\") pod \"multus-additional-cni-plugins-sl8sj\" (UID: \"a420108c-26d7-46d2-a3d5-3b354aa01af0\") " pod="openshift-multus/multus-additional-cni-plugins-sl8sj"
Jul 06 18:17:46.408796 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.302183    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5cfed2e4-8993-42f8-bc84-f9399b287729-host-run-netns\") pod \"ovnkube-node-9h479\" (UID: \"5cfed2e4-8993-42f8-bc84-f9399b287729\") " pod="openshift-ovn-kubernetes/ovnkube-node-9h479"
Jul 06 18:17:46.408796 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.302198    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/5cfed2e4-8993-42f8-bc84-f9399b287729-node-log\") pod \"ovnkube-node-9h479\" (UID: \"5cfed2e4-8993-42f8-bc84-f9399b287729\") " pod="openshift-ovn-kubernetes/ovnkube-node-9h479"
Jul 06 18:17:46.408796 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.302213    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5cfed2e4-8993-42f8-bc84-f9399b287729-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-9h479\" (UID: \"5cfed2e4-8993-42f8-bc84-f9399b287729\") " pod="openshift-ovn-kubernetes/ovnkube-node-9h479"
Jul 06 18:17:46.408796 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.302227    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d27a6ce1-0910-4c08-959a-550b8d468219-hosts-file\") pod \"node-resolver-x7rd5\" (UID: \"d27a6ce1-0910-4c08-959a-550b8d468219\") " pod="openshift-dns/node-resolver-x7rd5"
Jul 06 18:17:46.408796 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.302242    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/7b707150-01d0-47c3-b61e-a9a6f33465fc-run\") pod \"tuned-rttvb\" (UID: \"7b707150-01d0-47c3-b61e-a9a6f33465fc\") " pod="openshift-cluster-node-tuning-operator/tuned-rttvb"
Jul 06 18:17:46.409385 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.302257    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v2zzf\" (UniqueName: \"kubernetes.io/projected/c2ef9254-5ac3-45db-99a0-b5aff2a25d58-kube-api-access-v2zzf\") pod \"node-ca-kcr4l\" (UID: \"c2ef9254-5ac3-45db-99a0-b5aff2a25d58\") " pod="openshift-image-registry/node-ca-kcr4l"
Jul 06 18:17:46.409385 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.302273    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3b6b323e-eca7-4c68-87a7-0d48a9114a2c-original-pull-secret\") pod \"global-pull-secret-syncer-xzqch\" (UID: \"3b6b323e-eca7-4c68-87a7-0d48a9114a2c\") " pod="kube-system/global-pull-secret-syncer-xzqch"
Jul 06 18:17:46.409385 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.302288    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c5411d9d-cd87-44a4-a2b8-99acc89e5a2f-system-cni-dir\") pod \"multus-zk2x7\" (UID: \"c5411d9d-cd87-44a4-a2b8-99acc89e5a2f\") " pod="openshift-multus/multus-zk2x7"
Jul 06 18:17:46.409385 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.302335    2580 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 06 18:17:46.409385 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.302368    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5cfed2e4-8993-42f8-bc84-f9399b287729-host-run-netns\") pod \"ovnkube-node-9h479\" (UID: \"5cfed2e4-8993-42f8-bc84-f9399b287729\") " pod="openshift-ovn-kubernetes/ovnkube-node-9h479"
Jul 06 18:17:46.409385 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.302398    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/5cfed2e4-8993-42f8-bc84-f9399b287729-node-log\") pod \"ovnkube-node-9h479\" (UID: \"5cfed2e4-8993-42f8-bc84-f9399b287729\") " pod="openshift-ovn-kubernetes/ovnkube-node-9h479"
Jul 06 18:17:46.409385 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.302418    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5cfed2e4-8993-42f8-bc84-f9399b287729-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-9h479\" (UID: \"5cfed2e4-8993-42f8-bc84-f9399b287729\") " pod="openshift-ovn-kubernetes/ovnkube-node-9h479"
Jul 06 18:17:46.409385 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.302885    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/5cfed2e4-8993-42f8-bc84-f9399b287729-ovnkube-config\") pod \"ovnkube-node-9h479\" (UID: \"5cfed2e4-8993-42f8-bc84-f9399b287729\") " pod="openshift-ovn-kubernetes/ovnkube-node-9h479"
Jul 06 18:17:46.409385 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.302343    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f22387dc-2c30-4f80-8a80-3f6420c1c858-metrics-certs\") pod \"network-metrics-daemon-xjbzr\" (UID: \"f22387dc-2c30-4f80-8a80-3f6420c1c858\") " pod="openshift-multus/network-metrics-daemon-xjbzr"
Jul 06 18:17:46.409385 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.303179    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zst55\" (UniqueName: \"kubernetes.io/projected/d27a6ce1-0910-4c08-959a-550b8d468219-kube-api-access-zst55\") pod \"node-resolver-x7rd5\" (UID: \"d27a6ce1-0910-4c08-959a-550b8d468219\") " pod="openshift-dns/node-resolver-x7rd5"
Jul 06 18:17:46.409385 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.303212    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/7b707150-01d0-47c3-b61e-a9a6f33465fc-etc-sysctl-d\") pod \"tuned-rttvb\" (UID: \"7b707150-01d0-47c3-b61e-a9a6f33465fc\") " pod="openshift-cluster-node-tuning-operator/tuned-rttvb"
Jul 06 18:17:46.409385 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.303221    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/5cfed2e4-8993-42f8-bc84-f9399b287729-ovnkube-script-lib\") pod \"ovnkube-node-9h479\" (UID: \"5cfed2e4-8993-42f8-bc84-f9399b287729\") " pod="openshift-ovn-kubernetes/ovnkube-node-9h479"
Jul 06 18:17:46.409385 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.303240    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/7b707150-01d0-47c3-b61e-a9a6f33465fc-etc-systemd\") pod \"tuned-rttvb\" (UID: \"7b707150-01d0-47c3-b61e-a9a6f33465fc\") " pod="openshift-cluster-node-tuning-operator/tuned-rttvb"
Jul 06 18:17:46.409385 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.303264    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5cfed2e4-8993-42f8-bc84-f9399b287729-etc-openvswitch\") pod \"ovnkube-node-9h479\" (UID: \"5cfed2e4-8993-42f8-bc84-f9399b287729\") " pod="openshift-ovn-kubernetes/ovnkube-node-9h479"
Jul 06 18:17:46.409385 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.303268    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c2ef9254-5ac3-45db-99a0-b5aff2a25d58-host\") pod \"node-ca-kcr4l\" (UID: \"c2ef9254-5ac3-45db-99a0-b5aff2a25d58\") " pod="openshift-image-registry/node-ca-kcr4l"
Jul 06 18:17:46.409385 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.303293    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c5411d9d-cd87-44a4-a2b8-99acc89e5a2f-host-run-netns\") pod \"multus-zk2x7\" (UID: \"c5411d9d-cd87-44a4-a2b8-99acc89e5a2f\") " pod="openshift-multus/multus-zk2x7"
Jul 06 18:17:46.409385 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.303319    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c5411d9d-cd87-44a4-a2b8-99acc89e5a2f-host-var-lib-cni-bin\") pod \"multus-zk2x7\" (UID: \"c5411d9d-cd87-44a4-a2b8-99acc89e5a2f\") " pod="openshift-multus/multus-zk2x7"
Jul 06 18:17:46.409975 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.303362    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/c5411d9d-cd87-44a4-a2b8-99acc89e5a2f-multus-daemon-config\") pod \"multus-zk2x7\" (UID: \"c5411d9d-cd87-44a4-a2b8-99acc89e5a2f\") " pod="openshift-multus/multus-zk2x7"
Jul 06 18:17:46.409975 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.303426    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/5cfed2e4-8993-42f8-bc84-f9399b287729-systemd-units\") pod \"ovnkube-node-9h479\" (UID: \"5cfed2e4-8993-42f8-bc84-f9399b287729\") " pod="openshift-ovn-kubernetes/ovnkube-node-9h479"
Jul 06 18:17:46.409975 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.303446    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/7b707150-01d0-47c3-b61e-a9a6f33465fc-etc-tuned\") pod \"tuned-rttvb\" (UID: \"7b707150-01d0-47c3-b61e-a9a6f33465fc\") " pod="openshift-cluster-node-tuning-operator/tuned-rttvb"
Jul 06 18:17:46.409975 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.303482    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/c5411d9d-cd87-44a4-a2b8-99acc89e5a2f-host-run-k8s-cni-cncf-io\") pod \"multus-zk2x7\" (UID: \"c5411d9d-cd87-44a4-a2b8-99acc89e5a2f\") " pod="openshift-multus/multus-zk2x7"
Jul 06 18:17:46.409975 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.303515    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/5cfed2e4-8993-42f8-bc84-f9399b287729-systemd-units\") pod \"ovnkube-node-9h479\" (UID: \"5cfed2e4-8993-42f8-bc84-f9399b287729\") " pod="openshift-ovn-kubernetes/ovnkube-node-9h479"
Jul 06 18:17:46.409975 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.303541    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c5411d9d-cd87-44a4-a2b8-99acc89e5a2f-multus-conf-dir\") pod \"multus-zk2x7\" (UID: \"c5411d9d-cd87-44a4-a2b8-99acc89e5a2f\") " pod="openshift-multus/multus-zk2x7"
Jul 06 18:17:46.409975 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.305492    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/5cfed2e4-8993-42f8-bc84-f9399b287729-ovn-node-metrics-cert\") pod \"ovnkube-node-9h479\" (UID: \"5cfed2e4-8993-42f8-bc84-f9399b287729\") " pod="openshift-ovn-kubernetes/ovnkube-node-9h479"
Jul 06 18:17:46.409975 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:17:46.308238    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 06 18:17:46.409975 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:17:46.308252    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 06 18:17:46.409975 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:17:46.308261    2580 projected.go:196] Error preparing data for projected volume kube-api-access-knzwx for pod openshift-network-diagnostics/network-check-target-fp8fn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 18:17:46.409975 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:17:46.308303    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4e2f69c5-ef58-430a-8b74-eb53ce230c7e-kube-api-access-knzwx podName:4e2f69c5-ef58-430a-8b74-eb53ce230c7e nodeName:}" failed. No retries permitted until 2026-07-06 18:17:46.808289148 +0000 UTC m=+1.965687611 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-knzwx" (UniqueName: "kubernetes.io/projected/4e2f69c5-ef58-430a-8b74-eb53ce230c7e-kube-api-access-knzwx") pod "network-check-target-fp8fn" (UID: "4e2f69c5-ef58-430a-8b74-eb53ce230c7e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 18:17:46.409975 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.310896    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v5s29\" (UniqueName: \"kubernetes.io/projected/5cfed2e4-8993-42f8-bc84-f9399b287729-kube-api-access-v5s29\") pod \"ovnkube-node-9h479\" (UID: \"5cfed2e4-8993-42f8-bc84-f9399b287729\") " pod="openshift-ovn-kubernetes/ovnkube-node-9h479"
Jul 06 18:17:46.409975 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.319841    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-113.ec2.internal" event={"ID":"6eab86221f4cc923c73e223abc50211e","Type":"ContainerStarted","Data":"941babc7f25a3a02d53fc52f969f9ecb0115419e07a43b6bafc4e1356834cdd7"}
Jul 06 18:17:46.409975 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.321141    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-113.ec2.internal" event={"ID":"66d40220305fe03479bbe46175f2cd79","Type":"ContainerStarted","Data":"3e1411ce36a4936f930024a60eca4be30d939585778258bd8f30807681b87723"}
Jul 06 18:17:46.409975 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.404003    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/0474b033-5051-4df3-8135-4b05741835ac-device-dir\") pod \"aws-ebs-csi-driver-node-ztxm5\" (UID: \"0474b033-5051-4df3-8135-4b05741835ac\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ztxm5"
Jul 06 18:17:46.409975 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.404056    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-56m7v\" (UniqueName: \"kubernetes.io/projected/f22387dc-2c30-4f80-8a80-3f6420c1c858-kube-api-access-56m7v\") pod \"network-metrics-daemon-xjbzr\" (UID: \"f22387dc-2c30-4f80-8a80-3f6420c1c858\") " pod="openshift-multus/network-metrics-daemon-xjbzr"
Jul 06 18:17:46.409975 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.404100    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7b707150-01d0-47c3-b61e-a9a6f33465fc-lib-modules\") pod \"tuned-rttvb\" (UID: \"7b707150-01d0-47c3-b61e-a9a6f33465fc\") " pod="openshift-cluster-node-tuning-operator/tuned-rttvb"
Jul 06 18:17:46.410639 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.404130    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/3b6b323e-eca7-4c68-87a7-0d48a9114a2c-dbus\") pod \"global-pull-secret-syncer-xzqch\" (UID: \"3b6b323e-eca7-4c68-87a7-0d48a9114a2c\") " pod="kube-system/global-pull-secret-syncer-xzqch"
Jul 06 18:17:46.410639 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.404157    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7b707150-01d0-47c3-b61e-a9a6f33465fc-var-lib-kubelet\") pod \"tuned-rttvb\" (UID: \"7b707150-01d0-47c3-b61e-a9a6f33465fc\") " pod="openshift-cluster-node-tuning-operator/tuned-rttvb"
Jul 06 18:17:46.410639 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.404173    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7b707150-01d0-47c3-b61e-a9a6f33465fc-etc-kubernetes\") pod \"tuned-rttvb\" (UID: \"7b707150-01d0-47c3-b61e-a9a6f33465fc\") " pod="openshift-cluster-node-tuning-operator/tuned-rttvb"
Jul 06 18:17:46.410639 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.404190    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7b707150-01d0-47c3-b61e-a9a6f33465fc-tmp\") pod \"tuned-rttvb\" (UID: \"7b707150-01d0-47c3-b61e-a9a6f33465fc\") " pod="openshift-cluster-node-tuning-operator/tuned-rttvb"
Jul 06 18:17:46.410639 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.404205    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c5411d9d-cd87-44a4-a2b8-99acc89e5a2f-os-release\") pod \"multus-zk2x7\" (UID: \"c5411d9d-cd87-44a4-a2b8-99acc89e5a2f\") " pod="openshift-multus/multus-zk2x7"
Jul 06 18:17:46.410639 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.404219    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/0474b033-5051-4df3-8135-4b05741835ac-kubelet-dir\") pod \"aws-ebs-csi-driver-node-ztxm5\" (UID: \"0474b033-5051-4df3-8135-4b05741835ac\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ztxm5"
Jul 06 18:17:46.410639 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.404233    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a420108c-26d7-46d2-a3d5-3b354aa01af0-cnibin\") pod \"multus-additional-cni-plugins-sl8sj\" (UID: \"a420108c-26d7-46d2-a3d5-3b354aa01af0\") " pod="openshift-multus/multus-additional-cni-plugins-sl8sj"
Jul 06 18:17:46.410639 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.404250    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a420108c-26d7-46d2-a3d5-3b354aa01af0-tuning-conf-dir\") pod \"multus-additional-cni-plugins-sl8sj\" (UID: \"a420108c-26d7-46d2-a3d5-3b354aa01af0\") " pod="openshift-multus/multus-additional-cni-plugins-sl8sj"
Jul 06 18:17:46.410639 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.404266    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/3b6b323e-eca7-4c68-87a7-0d48a9114a2c-kubelet-config\") pod \"global-pull-secret-syncer-xzqch\" (UID: \"3b6b323e-eca7-4c68-87a7-0d48a9114a2c\") " pod="kube-system/global-pull-secret-syncer-xzqch"
Jul 06 18:17:46.410639 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.404281    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c5411d9d-cd87-44a4-a2b8-99acc89e5a2f-multus-cni-dir\") pod \"multus-zk2x7\" (UID: \"c5411d9d-cd87-44a4-a2b8-99acc89e5a2f\") " pod="openshift-multus/multus-zk2x7"
Jul 06 18:17:46.410639 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.404316    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bbbnf\" (UniqueName: \"kubernetes.io/projected/a420108c-26d7-46d2-a3d5-3b354aa01af0-kube-api-access-bbbnf\") pod \"multus-additional-cni-plugins-sl8sj\" (UID: \"a420108c-26d7-46d2-a3d5-3b354aa01af0\") " pod="openshift-multus/multus-additional-cni-plugins-sl8sj"
Jul 06 18:17:46.410639 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.404358    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7b707150-01d0-47c3-b61e-a9a6f33465fc-lib-modules\") pod \"tuned-rttvb\" (UID: \"7b707150-01d0-47c3-b61e-a9a6f33465fc\") " pod="openshift-cluster-node-tuning-operator/tuned-rttvb"
Jul 06 18:17:46.410639 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.404368    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/0474b033-5051-4df3-8135-4b05741835ac-device-dir\") pod \"aws-ebs-csi-driver-node-ztxm5\" (UID: \"0474b033-5051-4df3-8135-4b05741835ac\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ztxm5"
Jul 06 18:17:46.410639 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.404402    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/7b707150-01d0-47c3-b61e-a9a6f33465fc-etc-sysctl-conf\") pod \"tuned-rttvb\" (UID: \"7b707150-01d0-47c3-b61e-a9a6f33465fc\") " pod="openshift-cluster-node-tuning-operator/tuned-rttvb"
Jul 06 18:17:46.410639 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.404407    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7b707150-01d0-47c3-b61e-a9a6f33465fc-var-lib-kubelet\") pod \"tuned-rttvb\" (UID: \"7b707150-01d0-47c3-b61e-a9a6f33465fc\") " pod="openshift-cluster-node-tuning-operator/tuned-rttvb"
Jul 06 18:17:46.410639 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.404427    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c2ef9254-5ac3-45db-99a0-b5aff2a25d58-serviceca\") pod \"node-ca-kcr4l\" (UID: \"c2ef9254-5ac3-45db-99a0-b5aff2a25d58\") " pod="openshift-image-registry/node-ca-kcr4l"
Jul 06 18:17:46.410639 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.404444    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a420108c-26d7-46d2-a3d5-3b354aa01af0-cnibin\") pod \"multus-additional-cni-plugins-sl8sj\" (UID: \"a420108c-26d7-46d2-a3d5-3b354aa01af0\") " pod="openshift-multus/multus-additional-cni-plugins-sl8sj"
Jul 06 18:17:46.411232 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.404449    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/c5411d9d-cd87-44a4-a2b8-99acc89e5a2f-hostroot\") pod \"multus-zk2x7\" (UID: \"c5411d9d-cd87-44a4-a2b8-99acc89e5a2f\") " pod="openshift-multus/multus-zk2x7"
Jul 06 18:17:46.411232 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.404471    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/c5411d9d-cd87-44a4-a2b8-99acc89e5a2f-host-run-multus-certs\") pod \"multus-zk2x7\" (UID: \"c5411d9d-cd87-44a4-a2b8-99acc89e5a2f\") " pod="openshift-multus/multus-zk2x7"
Jul 06 18:17:46.411232 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.404494    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jfnjw\" (UniqueName: \"kubernetes.io/projected/af914b85-02d3-423a-aed5-e96b26e6c1c1-kube-api-access-jfnjw\") pod \"iptables-alerter-vz7bl\" (UID: \"af914b85-02d3-423a-aed5-e96b26e6c1c1\") " pod="openshift-network-operator/iptables-alerter-vz7bl"
Jul 06 18:17:46.411232 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.404506    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7b707150-01d0-47c3-b61e-a9a6f33465fc-etc-kubernetes\") pod \"tuned-rttvb\" (UID: \"7b707150-01d0-47c3-b61e-a9a6f33465fc\") " pod="openshift-cluster-node-tuning-operator/tuned-rttvb"
Jul 06 18:17:46.411232 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.404515    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a420108c-26d7-46d2-a3d5-3b354aa01af0-os-release\") pod \"multus-additional-cni-plugins-sl8sj\" (UID: \"a420108c-26d7-46d2-a3d5-3b354aa01af0\") " pod="openshift-multus/multus-additional-cni-plugins-sl8sj"
Jul 06 18:17:46.411232 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.404544    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d27a6ce1-0910-4c08-959a-550b8d468219-hosts-file\") pod \"node-resolver-x7rd5\" (UID: \"d27a6ce1-0910-4c08-959a-550b8d468219\") " pod="openshift-dns/node-resolver-x7rd5"
Jul 06 18:17:46.411232 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.404564    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/7b707150-01d0-47c3-b61e-a9a6f33465fc-run\") pod \"tuned-rttvb\" (UID: \"7b707150-01d0-47c3-b61e-a9a6f33465fc\") " pod="openshift-cluster-node-tuning-operator/tuned-rttvb"
Jul 06 18:17:46.411232 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.404586    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v2zzf\" (UniqueName: \"kubernetes.io/projected/c2ef9254-5ac3-45db-99a0-b5aff2a25d58-kube-api-access-v2zzf\") pod \"node-ca-kcr4l\" (UID: \"c2ef9254-5ac3-45db-99a0-b5aff2a25d58\") " pod="openshift-image-registry/node-ca-kcr4l"
Jul 06 18:17:46.411232 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.404609    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3b6b323e-eca7-4c68-87a7-0d48a9114a2c-original-pull-secret\") pod \"global-pull-secret-syncer-xzqch\" (UID: \"3b6b323e-eca7-4c68-87a7-0d48a9114a2c\") " pod="kube-system/global-pull-secret-syncer-xzqch"
Jul 06 18:17:46.411232 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.404626    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a420108c-26d7-46d2-a3d5-3b354aa01af0-tuning-conf-dir\") pod \"multus-additional-cni-plugins-sl8sj\" (UID: \"a420108c-26d7-46d2-a3d5-3b354aa01af0\") " pod="openshift-multus/multus-additional-cni-plugins-sl8sj"
Jul 06 18:17:46.411232 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.404630    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c5411d9d-cd87-44a4-a2b8-99acc89e5a2f-system-cni-dir\") pod \"multus-zk2x7\" (UID: \"c5411d9d-cd87-44a4-a2b8-99acc89e5a2f\") " pod="openshift-multus/multus-zk2x7"
Jul 06 18:17:46.411232 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.404652    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f22387dc-2c30-4f80-8a80-3f6420c1c858-metrics-certs\") pod \"network-metrics-daemon-xjbzr\" (UID: \"f22387dc-2c30-4f80-8a80-3f6420c1c858\") " pod="openshift-multus/network-metrics-daemon-xjbzr"
Jul 06 18:17:46.411232 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.404672    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zst55\" (UniqueName: \"kubernetes.io/projected/d27a6ce1-0910-4c08-959a-550b8d468219-kube-api-access-zst55\") pod \"node-resolver-x7rd5\" (UID: \"d27a6ce1-0910-4c08-959a-550b8d468219\") " pod="openshift-dns/node-resolver-x7rd5"
Jul 06 18:17:46.411232 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.404762    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/7b707150-01d0-47c3-b61e-a9a6f33465fc-etc-sysctl-d\") pod \"tuned-rttvb\" (UID: \"7b707150-01d0-47c3-b61e-a9a6f33465fc\") " pod="openshift-cluster-node-tuning-operator/tuned-rttvb"
Jul 06 18:17:46.411232 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.404781    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c5411d9d-cd87-44a4-a2b8-99acc89e5a2f-multus-cni-dir\") pod \"multus-zk2x7\" (UID: \"c5411d9d-cd87-44a4-a2b8-99acc89e5a2f\") " pod="openshift-multus/multus-zk2x7"
Jul 06 18:17:46.411232 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.404786    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/7b707150-01d0-47c3-b61e-a9a6f33465fc-etc-systemd\") pod \"tuned-rttvb\" (UID: \"7b707150-01d0-47c3-b61e-a9a6f33465fc\") " pod="openshift-cluster-node-tuning-operator/tuned-rttvb"
Jul 06 18:17:46.411232 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.404807    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c2ef9254-5ac3-45db-99a0-b5aff2a25d58-host\") pod \"node-ca-kcr4l\" (UID: \"c2ef9254-5ac3-45db-99a0-b5aff2a25d58\") " pod="openshift-image-registry/node-ca-kcr4l"
Jul 06 18:17:46.411232 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.404816    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d27a6ce1-0910-4c08-959a-550b8d468219-hosts-file\") pod \"node-resolver-x7rd5\" (UID: \"d27a6ce1-0910-4c08-959a-550b8d468219\") " pod="openshift-dns/node-resolver-x7rd5"
Jul 06 18:17:46.412253 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.404833    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c5411d9d-cd87-44a4-a2b8-99acc89e5a2f-host-run-netns\") pod \"multus-zk2x7\" (UID: \"c5411d9d-cd87-44a4-a2b8-99acc89e5a2f\") " pod="openshift-multus/multus-zk2x7"
Jul 06 18:17:46.412253 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.404856    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c5411d9d-cd87-44a4-a2b8-99acc89e5a2f-host-var-lib-cni-bin\") pod \"multus-zk2x7\" (UID: \"c5411d9d-cd87-44a4-a2b8-99acc89e5a2f\") " pod="openshift-multus/multus-zk2x7"
Jul 06 18:17:46.412253 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.404877    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/c5411d9d-cd87-44a4-a2b8-99acc89e5a2f-multus-daemon-config\") pod \"multus-zk2x7\" (UID: \"c5411d9d-cd87-44a4-a2b8-99acc89e5a2f\") " pod="openshift-multus/multus-zk2x7"
Jul 06 18:17:46.412253 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.404900    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/7b707150-01d0-47c3-b61e-a9a6f33465fc-etc-tuned\") pod \"tuned-rttvb\" (UID: \"7b707150-01d0-47c3-b61e-a9a6f33465fc\") " pod="openshift-cluster-node-tuning-operator/tuned-rttvb"
Jul 06 18:17:46.412253 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.404960    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/c5411d9d-cd87-44a4-a2b8-99acc89e5a2f-host-run-k8s-cni-cncf-io\") pod \"multus-zk2x7\" (UID: \"c5411d9d-cd87-44a4-a2b8-99acc89e5a2f\") " pod="openshift-multus/multus-zk2x7"
Jul 06 18:17:46.412253 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.404981    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c5411d9d-cd87-44a4-a2b8-99acc89e5a2f-multus-conf-dir\") pod \"multus-zk2x7\" (UID: \"c5411d9d-cd87-44a4-a2b8-99acc89e5a2f\") " pod="openshift-multus/multus-zk2x7"
Jul 06 18:17:46.412253 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.405003    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/0474b033-5051-4df3-8135-4b05741835ac-registration-dir\") pod \"aws-ebs-csi-driver-node-ztxm5\" (UID: \"0474b033-5051-4df3-8135-4b05741835ac\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ztxm5"
Jul 06 18:17:46.412253 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.405040    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/0474b033-5051-4df3-8135-4b05741835ac-etc-selinux\") pod \"aws-ebs-csi-driver-node-ztxm5\" (UID: \"0474b033-5051-4df3-8135-4b05741835ac\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ztxm5"
Jul 06 18:17:46.412253 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.405065    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a420108c-26d7-46d2-a3d5-3b354aa01af0-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-sl8sj\" (UID: \"a420108c-26d7-46d2-a3d5-3b354aa01af0\") " pod="openshift-multus/multus-additional-cni-plugins-sl8sj"
Jul 06 18:17:46.412253 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.405092    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/7b707150-01d0-47c3-b61e-a9a6f33465fc-etc-modprobe-d\") pod \"tuned-rttvb\" (UID: \"7b707150-01d0-47c3-b61e-a9a6f33465fc\") " pod="openshift-cluster-node-tuning-operator/tuned-rttvb"
Jul 06 18:17:46.412253 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.405100    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/0474b033-5051-4df3-8135-4b05741835ac-kubelet-dir\") pod \"aws-ebs-csi-driver-node-ztxm5\" (UID: \"0474b033-5051-4df3-8135-4b05741835ac\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ztxm5"
Jul 06 18:17:46.412253 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.405113    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c5411d9d-cd87-44a4-a2b8-99acc89e5a2f-etc-kubernetes\") pod \"multus-zk2x7\" (UID: \"c5411d9d-cd87-44a4-a2b8-99acc89e5a2f\") " pod="openshift-multus/multus-zk2x7"
Jul 06 18:17:46.412253 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.405137    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/af914b85-02d3-423a-aed5-e96b26e6c1c1-iptables-alerter-script\") pod \"iptables-alerter-vz7bl\" (UID: \"af914b85-02d3-423a-aed5-e96b26e6c1c1\") " pod="openshift-network-operator/iptables-alerter-vz7bl"
Jul 06 18:17:46.412253 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.405160    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c5411d9d-cd87-44a4-a2b8-99acc89e5a2f-host-run-netns\") pod \"multus-zk2x7\" (UID: \"c5411d9d-cd87-44a4-a2b8-99acc89e5a2f\") " pod="openshift-multus/multus-zk2x7"
Jul 06 18:17:46.412253 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.405224    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/7b707150-01d0-47c3-b61e-a9a6f33465fc-run\") pod \"tuned-rttvb\" (UID: \"7b707150-01d0-47c3-b61e-a9a6f33465fc\") " pod="openshift-cluster-node-tuning-operator/tuned-rttvb"
Jul 06 18:17:46.412253 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.405377    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l4rrx\" (UniqueName: \"kubernetes.io/projected/0474b033-5051-4df3-8135-4b05741835ac-kube-api-access-l4rrx\") pod \"aws-ebs-csi-driver-node-ztxm5\" (UID: \"0474b033-5051-4df3-8135-4b05741835ac\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ztxm5"
Jul 06 18:17:46.412253 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.405401    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a420108c-26d7-46d2-a3d5-3b354aa01af0-system-cni-dir\") pod \"multus-additional-cni-plugins-sl8sj\" (UID: \"a420108c-26d7-46d2-a3d5-3b354aa01af0\") " pod="openshift-multus/multus-additional-cni-plugins-sl8sj"
Jul 06 18:17:46.413264 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.405423    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a420108c-26d7-46d2-a3d5-3b354aa01af0-cni-binary-copy\") pod \"multus-additional-cni-plugins-sl8sj\" (UID: \"a420108c-26d7-46d2-a3d5-3b354aa01af0\") " pod="openshift-multus/multus-additional-cni-plugins-sl8sj"
Jul 06 18:17:46.413264 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.405451    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/3b6b323e-eca7-4c68-87a7-0d48a9114a2c-dbus\") pod \"global-pull-secret-syncer-xzqch\" (UID: \"3b6b323e-eca7-4c68-87a7-0d48a9114a2c\") " pod="kube-system/global-pull-secret-syncer-xzqch"
Jul 06 18:17:46.413264 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.405467    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/0474b033-5051-4df3-8135-4b05741835ac-sys-fs\") pod \"aws-ebs-csi-driver-node-ztxm5\" (UID: \"0474b033-5051-4df3-8135-4b05741835ac\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ztxm5"
Jul 06 18:17:46.413264 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.405491    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f117fdd5-8a61-484e-8ebc-668f89723813-konnectivity-ca\") pod \"konnectivity-agent-b7jm9\" (UID: \"f117fdd5-8a61-484e-8ebc-668f89723813\") " pod="kube-system/konnectivity-agent-b7jm9"
Jul 06 18:17:46.413264 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.405517    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7b707150-01d0-47c3-b61e-a9a6f33465fc-host\") pod \"tuned-rttvb\" (UID: \"7b707150-01d0-47c3-b61e-a9a6f33465fc\") " pod="openshift-cluster-node-tuning-operator/tuned-rttvb"
Jul 06 18:17:46.413264 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.405533    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c5411d9d-cd87-44a4-a2b8-99acc89e5a2f-os-release\") pod \"multus-zk2x7\" (UID: \"c5411d9d-cd87-44a4-a2b8-99acc89e5a2f\") " pod="openshift-multus/multus-zk2x7"
Jul 06 18:17:46.413264 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.405580    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7b707150-01d0-47c3-b61e-a9a6f33465fc-host\") pod \"tuned-rttvb\" (UID: \"7b707150-01d0-47c3-b61e-a9a6f33465fc\") " pod="openshift-cluster-node-tuning-operator/tuned-rttvb"
Jul 06 18:17:46.413264 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.405590    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/3b6b323e-eca7-4c68-87a7-0d48a9114a2c-kubelet-config\") pod \"global-pull-secret-syncer-xzqch\" (UID: \"3b6b323e-eca7-4c68-87a7-0d48a9114a2c\") " pod="kube-system/global-pull-secret-syncer-xzqch"
Jul 06 18:17:46.413264 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.405608    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fdvzt\" (UniqueName: \"kubernetes.io/projected/7b707150-01d0-47c3-b61e-a9a6f33465fc-kube-api-access-fdvzt\") pod \"tuned-rttvb\" (UID: \"7b707150-01d0-47c3-b61e-a9a6f33465fc\") " pod="openshift-cluster-node-tuning-operator/tuned-rttvb"
Jul 06 18:17:46.413264 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.405634    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c5411d9d-cd87-44a4-a2b8-99acc89e5a2f-cnibin\") pod \"multus-zk2x7\" (UID: \"c5411d9d-cd87-44a4-a2b8-99acc89e5a2f\") " pod="openshift-multus/multus-zk2x7"
Jul 06 18:17:46.413264 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.405657    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/c5411d9d-cd87-44a4-a2b8-99acc89e5a2f-multus-socket-dir-parent\") pod \"multus-zk2x7\" (UID: \"c5411d9d-cd87-44a4-a2b8-99acc89e5a2f\") " pod="openshift-multus/multus-zk2x7"
Jul 06 18:17:46.413264 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.405693    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c5411d9d-cd87-44a4-a2b8-99acc89e5a2f-host-var-lib-cni-bin\") pod \"multus-zk2x7\" (UID: \"c5411d9d-cd87-44a4-a2b8-99acc89e5a2f\") " pod="openshift-multus/multus-zk2x7"
Jul 06 18:17:46.413264 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.405707    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/c5411d9d-cd87-44a4-a2b8-99acc89e5a2f-host-var-lib-cni-multus\") pod \"multus-zk2x7\" (UID: \"c5411d9d-cd87-44a4-a2b8-99acc89e5a2f\") " pod="openshift-multus/multus-zk2x7"
Jul 06 18:17:46.413264 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.405730    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c5411d9d-cd87-44a4-a2b8-99acc89e5a2f-host-var-lib-kubelet\") pod \"multus-zk2x7\" (UID: \"c5411d9d-cd87-44a4-a2b8-99acc89e5a2f\") " pod="openshift-multus/multus-zk2x7"
Jul 06 18:17:46.413264 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.405753    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kf4f6\" (UniqueName: \"kubernetes.io/projected/c5411d9d-cd87-44a4-a2b8-99acc89e5a2f-kube-api-access-kf4f6\") pod \"multus-zk2x7\" (UID: \"c5411d9d-cd87-44a4-a2b8-99acc89e5a2f\") " pod="openshift-multus/multus-zk2x7"
Jul 06 18:17:46.413264 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.405775    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f117fdd5-8a61-484e-8ebc-668f89723813-agent-certs\") pod \"konnectivity-agent-b7jm9\" (UID: \"f117fdd5-8a61-484e-8ebc-668f89723813\") " pod="kube-system/konnectivity-agent-b7jm9"
Jul 06 18:17:46.413264 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.405815    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d27a6ce1-0910-4c08-959a-550b8d468219-tmp-dir\") pod \"node-resolver-x7rd5\" (UID: \"d27a6ce1-0910-4c08-959a-550b8d468219\") " pod="openshift-dns/node-resolver-x7rd5"
Jul 06 18:17:46.413264 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.405848    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/7b707150-01d0-47c3-b61e-a9a6f33465fc-etc-sysctl-d\") pod \"tuned-rttvb\" (UID: \"7b707150-01d0-47c3-b61e-a9a6f33465fc\") " pod="openshift-cluster-node-tuning-operator/tuned-rttvb"
Jul 06 18:17:46.414164 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.405902    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/7b707150-01d0-47c3-b61e-a9a6f33465fc-etc-sysconfig\") pod \"tuned-rttvb\" (UID: \"7b707150-01d0-47c3-b61e-a9a6f33465fc\") " pod="openshift-cluster-node-tuning-operator/tuned-rttvb"
Jul 06 18:17:46.414164 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.405918    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c2ef9254-5ac3-45db-99a0-b5aff2a25d58-serviceca\") pod \"node-ca-kcr4l\" (UID: \"c2ef9254-5ac3-45db-99a0-b5aff2a25d58\") " pod="openshift-image-registry/node-ca-kcr4l"
Jul 06 18:17:46.414164 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.405947    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/7b707150-01d0-47c3-b61e-a9a6f33465fc-etc-systemd\") pod \"tuned-rttvb\" (UID: \"7b707150-01d0-47c3-b61e-a9a6f33465fc\") " pod="openshift-cluster-node-tuning-operator/tuned-rttvb"
Jul 06 18:17:46.414164 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.405970    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/c5411d9d-cd87-44a4-a2b8-99acc89e5a2f-hostroot\") pod \"multus-zk2x7\" (UID: \"c5411d9d-cd87-44a4-a2b8-99acc89e5a2f\") " pod="openshift-multus/multus-zk2x7"
Jul 06 18:17:46.414164 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.405990    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c2ef9254-5ac3-45db-99a0-b5aff2a25d58-host\") pod \"node-ca-kcr4l\" (UID: \"c2ef9254-5ac3-45db-99a0-b5aff2a25d58\") " pod="openshift-image-registry/node-ca-kcr4l"
Jul 06 18:17:46.414164 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.406001    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/c5411d9d-cd87-44a4-a2b8-99acc89e5a2f-host-run-multus-certs\") pod \"multus-zk2x7\" (UID: \"c5411d9d-cd87-44a4-a2b8-99acc89e5a2f\") " pod="openshift-multus/multus-zk2x7"
Jul 06 18:17:46.414164 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.405841    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/7b707150-01d0-47c3-b61e-a9a6f33465fc-etc-sysconfig\") pod \"tuned-rttvb\" (UID: \"7b707150-01d0-47c3-b61e-a9a6f33465fc\") " pod="openshift-cluster-node-tuning-operator/tuned-rttvb"
Jul 06 18:17:46.414164 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.406053    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7b707150-01d0-47c3-b61e-a9a6f33465fc-sys\") pod \"tuned-rttvb\" (UID: \"7b707150-01d0-47c3-b61e-a9a6f33465fc\") " pod="openshift-cluster-node-tuning-operator/tuned-rttvb"
Jul 06 18:17:46.414164 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.406105    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c5411d9d-cd87-44a4-a2b8-99acc89e5a2f-cni-binary-copy\") pod \"multus-zk2x7\" (UID: \"c5411d9d-cd87-44a4-a2b8-99acc89e5a2f\") " pod="openshift-multus/multus-zk2x7"
Jul 06 18:17:46.414164 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.406132    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/af914b85-02d3-423a-aed5-e96b26e6c1c1-host-slash\") pod \"iptables-alerter-vz7bl\" (UID: \"af914b85-02d3-423a-aed5-e96b26e6c1c1\") " pod="openshift-network-operator/iptables-alerter-vz7bl"
Jul 06 18:17:46.414164 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.406159    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/0474b033-5051-4df3-8135-4b05741835ac-socket-dir\") pod \"aws-ebs-csi-driver-node-ztxm5\" (UID: \"0474b033-5051-4df3-8135-4b05741835ac\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ztxm5"
Jul 06 18:17:46.414164 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.406170    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/c5411d9d-cd87-44a4-a2b8-99acc89e5a2f-multus-daemon-config\") pod \"multus-zk2x7\" (UID: \"c5411d9d-cd87-44a4-a2b8-99acc89e5a2f\") " pod="openshift-multus/multus-zk2x7"
Jul 06 18:17:46.414164 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.406201    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a420108c-26d7-46d2-a3d5-3b354aa01af0-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-sl8sj\" (UID: \"a420108c-26d7-46d2-a3d5-3b354aa01af0\") " pod="openshift-multus/multus-additional-cni-plugins-sl8sj"
Jul 06 18:17:46.414164 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.406256    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a420108c-26d7-46d2-a3d5-3b354aa01af0-os-release\") pod \"multus-additional-cni-plugins-sl8sj\" (UID: \"a420108c-26d7-46d2-a3d5-3b354aa01af0\") " pod="openshift-multus/multus-additional-cni-plugins-sl8sj"
Jul 06 18:17:46.414164 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.406368    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/0474b033-5051-4df3-8135-4b05741835ac-socket-dir\") pod \"aws-ebs-csi-driver-node-ztxm5\" (UID: \"0474b033-5051-4df3-8135-4b05741835ac\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ztxm5"
Jul 06 18:17:46.414164 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.406416    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7b707150-01d0-47c3-b61e-a9a6f33465fc-sys\") pod \"tuned-rttvb\" (UID: \"7b707150-01d0-47c3-b61e-a9a6f33465fc\") " pod="openshift-cluster-node-tuning-operator/tuned-rttvb"
Jul 06 18:17:46.414164 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.406417    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a420108c-26d7-46d2-a3d5-3b354aa01af0-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-sl8sj\" (UID: \"a420108c-26d7-46d2-a3d5-3b354aa01af0\") " pod="openshift-multus/multus-additional-cni-plugins-sl8sj"
Jul 06 18:17:46.414164 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.406501    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/7b707150-01d0-47c3-b61e-a9a6f33465fc-etc-modprobe-d\") pod \"tuned-rttvb\" (UID: \"7b707150-01d0-47c3-b61e-a9a6f33465fc\") " pod="openshift-cluster-node-tuning-operator/tuned-rttvb"
Jul 06 18:17:46.414736 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.406546    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c5411d9d-cd87-44a4-a2b8-99acc89e5a2f-etc-kubernetes\") pod \"multus-zk2x7\" (UID: \"c5411d9d-cd87-44a4-a2b8-99acc89e5a2f\") " pod="openshift-multus/multus-zk2x7"
Jul 06 18:17:46.414736 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.406643    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a420108c-26d7-46d2-a3d5-3b354aa01af0-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-sl8sj\" (UID: \"a420108c-26d7-46d2-a3d5-3b354aa01af0\") " pod="openshift-multus/multus-additional-cni-plugins-sl8sj"
Jul 06 18:17:46.414736 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.406700    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/af914b85-02d3-423a-aed5-e96b26e6c1c1-host-slash\") pod \"iptables-alerter-vz7bl\" (UID: \"af914b85-02d3-423a-aed5-e96b26e6c1c1\") " pod="openshift-network-operator/iptables-alerter-vz7bl"
Jul 06 18:17:46.414736 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.406744    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c5411d9d-cd87-44a4-a2b8-99acc89e5a2f-cnibin\") pod \"multus-zk2x7\" (UID: \"c5411d9d-cd87-44a4-a2b8-99acc89e5a2f\") " pod="openshift-multus/multus-zk2x7"
Jul 06 18:17:46.414736 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.406838    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/c5411d9d-cd87-44a4-a2b8-99acc89e5a2f-host-run-k8s-cni-cncf-io\") pod \"multus-zk2x7\" (UID: \"c5411d9d-cd87-44a4-a2b8-99acc89e5a2f\") " pod="openshift-multus/multus-zk2x7"
Jul 06 18:17:46.414736 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.406895    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c5411d9d-cd87-44a4-a2b8-99acc89e5a2f-multus-conf-dir\") pod \"multus-zk2x7\" (UID: \"c5411d9d-cd87-44a4-a2b8-99acc89e5a2f\") " pod="openshift-multus/multus-zk2x7"
Jul 06 18:17:46.414736 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.406918    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c5411d9d-cd87-44a4-a2b8-99acc89e5a2f-cni-binary-copy\") pod \"multus-zk2x7\" (UID: \"c5411d9d-cd87-44a4-a2b8-99acc89e5a2f\") " pod="openshift-multus/multus-zk2x7"
Jul 06 18:17:46.414736 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.406949    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/0474b033-5051-4df3-8135-4b05741835ac-registration-dir\") pod \"aws-ebs-csi-driver-node-ztxm5\" (UID: \"0474b033-5051-4df3-8135-4b05741835ac\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ztxm5"
Jul 06 18:17:46.414736 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.406985    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a420108c-26d7-46d2-a3d5-3b354aa01af0-system-cni-dir\") pod \"multus-additional-cni-plugins-sl8sj\" (UID: \"a420108c-26d7-46d2-a3d5-3b354aa01af0\") " pod="openshift-multus/multus-additional-cni-plugins-sl8sj"
Jul 06 18:17:46.414736 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.405532    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c5411d9d-cd87-44a4-a2b8-99acc89e5a2f-system-cni-dir\") pod \"multus-zk2x7\" (UID: \"c5411d9d-cd87-44a4-a2b8-99acc89e5a2f\") " pod="openshift-multus/multus-zk2x7"
Jul 06 18:17:46.414736 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.407051    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7b707150-01d0-47c3-b61e-a9a6f33465fc-tmp\") pod \"tuned-rttvb\" (UID: \"7b707150-01d0-47c3-b61e-a9a6f33465fc\") " pod="openshift-cluster-node-tuning-operator/tuned-rttvb"
Jul 06 18:17:46.414736 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:17:46.405661    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 18:17:46.414736 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:17:46.407120    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 06 18:17:46.414736 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:17:46.407134    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f22387dc-2c30-4f80-8a80-3f6420c1c858-metrics-certs podName:f22387dc-2c30-4f80-8a80-3f6420c1c858 nodeName:}" failed. No retries permitted until 2026-07-06 18:17:46.907118474 +0000 UTC m=+2.064516941 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f22387dc-2c30-4f80-8a80-3f6420c1c858-metrics-certs") pod "network-metrics-daemon-xjbzr" (UID: "f22387dc-2c30-4f80-8a80-3f6420c1c858") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 18:17:46.414736 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.407150    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/c5411d9d-cd87-44a4-a2b8-99acc89e5a2f-multus-socket-dir-parent\") pod \"multus-zk2x7\" (UID: \"c5411d9d-cd87-44a4-a2b8-99acc89e5a2f\") " pod="openshift-multus/multus-zk2x7"
Jul 06 18:17:46.414736 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.407037    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/af914b85-02d3-423a-aed5-e96b26e6c1c1-iptables-alerter-script\") pod \"iptables-alerter-vz7bl\" (UID: \"af914b85-02d3-423a-aed5-e96b26e6c1c1\") " pod="openshift-network-operator/iptables-alerter-vz7bl"
Jul 06 18:17:46.414736 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:17:46.407169    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b6b323e-eca7-4c68-87a7-0d48a9114a2c-original-pull-secret podName:3b6b323e-eca7-4c68-87a7-0d48a9114a2c nodeName:}" failed. No retries permitted until 2026-07-06 18:17:46.907153354 +0000 UTC m=+2.064551833 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3b6b323e-eca7-4c68-87a7-0d48a9114a2c-original-pull-secret") pod "global-pull-secret-syncer-xzqch" (UID: "3b6b323e-eca7-4c68-87a7-0d48a9114a2c") : object "kube-system"/"original-pull-secret" not registered
Jul 06 18:17:46.415413 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.407171    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/7b707150-01d0-47c3-b61e-a9a6f33465fc-etc-sysctl-conf\") pod \"tuned-rttvb\" (UID: \"7b707150-01d0-47c3-b61e-a9a6f33465fc\") " pod="openshift-cluster-node-tuning-operator/tuned-rttvb"
Jul 06 18:17:46.415413 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.407218    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c5411d9d-cd87-44a4-a2b8-99acc89e5a2f-host-var-lib-kubelet\") pod \"multus-zk2x7\" (UID: \"c5411d9d-cd87-44a4-a2b8-99acc89e5a2f\") " pod="openshift-multus/multus-zk2x7"
Jul 06 18:17:46.415413 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.407225    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/c5411d9d-cd87-44a4-a2b8-99acc89e5a2f-host-var-lib-cni-multus\") pod \"multus-zk2x7\" (UID: \"c5411d9d-cd87-44a4-a2b8-99acc89e5a2f\") " pod="openshift-multus/multus-zk2x7"
Jul 06 18:17:46.415413 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.407288    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/0474b033-5051-4df3-8135-4b05741835ac-sys-fs\") pod \"aws-ebs-csi-driver-node-ztxm5\" (UID: \"0474b033-5051-4df3-8135-4b05741835ac\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ztxm5"
Jul 06 18:17:46.415413 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.407052    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/0474b033-5051-4df3-8135-4b05741835ac-etc-selinux\") pod \"aws-ebs-csi-driver-node-ztxm5\" (UID: \"0474b033-5051-4df3-8135-4b05741835ac\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ztxm5"
Jul 06 18:17:46.415413 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.407401    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a420108c-26d7-46d2-a3d5-3b354aa01af0-cni-binary-copy\") pod \"multus-additional-cni-plugins-sl8sj\" (UID: \"a420108c-26d7-46d2-a3d5-3b354aa01af0\") " pod="openshift-multus/multus-additional-cni-plugins-sl8sj"
Jul 06 18:17:46.415413 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.407498    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d27a6ce1-0910-4c08-959a-550b8d468219-tmp-dir\") pod \"node-resolver-x7rd5\" (UID: \"d27a6ce1-0910-4c08-959a-550b8d468219\") " pod="openshift-dns/node-resolver-x7rd5"
Jul 06 18:17:46.415413 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.407763    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f117fdd5-8a61-484e-8ebc-668f89723813-konnectivity-ca\") pod \"konnectivity-agent-b7jm9\" (UID: \"f117fdd5-8a61-484e-8ebc-668f89723813\") " pod="kube-system/konnectivity-agent-b7jm9"
Jul 06 18:17:46.415413 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.408325    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/7b707150-01d0-47c3-b61e-a9a6f33465fc-etc-tuned\") pod \"tuned-rttvb\" (UID: \"7b707150-01d0-47c3-b61e-a9a6f33465fc\") " pod="openshift-cluster-node-tuning-operator/tuned-rttvb"
Jul 06 18:17:46.415413 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.409288    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f117fdd5-8a61-484e-8ebc-668f89723813-agent-certs\") pod \"konnectivity-agent-b7jm9\" (UID: \"f117fdd5-8a61-484e-8ebc-668f89723813\") " pod="kube-system/konnectivity-agent-b7jm9"
Jul 06 18:17:46.415413 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.413164    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v2zzf\" (UniqueName: \"kubernetes.io/projected/c2ef9254-5ac3-45db-99a0-b5aff2a25d58-kube-api-access-v2zzf\") pod \"node-ca-kcr4l\" (UID: \"c2ef9254-5ac3-45db-99a0-b5aff2a25d58\") " pod="openshift-image-registry/node-ca-kcr4l"
Jul 06 18:17:46.415413 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.413489    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-56m7v\" (UniqueName: \"kubernetes.io/projected/f22387dc-2c30-4f80-8a80-3f6420c1c858-kube-api-access-56m7v\") pod \"network-metrics-daemon-xjbzr\" (UID: \"f22387dc-2c30-4f80-8a80-3f6420c1c858\") " pod="openshift-multus/network-metrics-daemon-xjbzr"
Jul 06 18:17:46.415413 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.413676    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zst55\" (UniqueName: \"kubernetes.io/projected/d27a6ce1-0910-4c08-959a-550b8d468219-kube-api-access-zst55\") pod \"node-resolver-x7rd5\" (UID: \"d27a6ce1-0910-4c08-959a-550b8d468219\") " pod="openshift-dns/node-resolver-x7rd5"
Jul 06 18:17:46.415413 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.414081    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jfnjw\" (UniqueName: \"kubernetes.io/projected/af914b85-02d3-423a-aed5-e96b26e6c1c1-kube-api-access-jfnjw\") pod \"iptables-alerter-vz7bl\" (UID: \"af914b85-02d3-423a-aed5-e96b26e6c1c1\") " pod="openshift-network-operator/iptables-alerter-vz7bl"
Jul 06 18:17:46.415413 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.414341    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bbbnf\" (UniqueName: \"kubernetes.io/projected/a420108c-26d7-46d2-a3d5-3b354aa01af0-kube-api-access-bbbnf\") pod \"multus-additional-cni-plugins-sl8sj\" (UID: \"a420108c-26d7-46d2-a3d5-3b354aa01af0\") " pod="openshift-multus/multus-additional-cni-plugins-sl8sj"
Jul 06 18:17:46.415413 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.415155    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l4rrx\" (UniqueName: \"kubernetes.io/projected/0474b033-5051-4df3-8135-4b05741835ac-kube-api-access-l4rrx\") pod \"aws-ebs-csi-driver-node-ztxm5\" (UID: \"0474b033-5051-4df3-8135-4b05741835ac\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ztxm5"
Jul 06 18:17:46.415901 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.415547    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kf4f6\" (UniqueName: \"kubernetes.io/projected/c5411d9d-cd87-44a4-a2b8-99acc89e5a2f-kube-api-access-kf4f6\") pod \"multus-zk2x7\" (UID: \"c5411d9d-cd87-44a4-a2b8-99acc89e5a2f\") " pod="openshift-multus/multus-zk2x7"
Jul 06 18:17:46.416051 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.415909    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fdvzt\" (UniqueName: \"kubernetes.io/projected/7b707150-01d0-47c3-b61e-a9a6f33465fc-kube-api-access-fdvzt\") pod \"tuned-rttvb\" (UID: \"7b707150-01d0-47c3-b61e-a9a6f33465fc\") " pod="openshift-cluster-node-tuning-operator/tuned-rttvb"
Jul 06 18:17:46.561181 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.525640    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-9h479"
Jul 06 18:17:46.561181 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:46.533987    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5cfed2e4_8993_42f8_bc84_f9399b287729.slice/crio-165b1b9c64b16eb4e2bc10312dba48eaaf4189c1abc3c97a0d26144f8e9aee72 WatchSource:0}: Error finding container 165b1b9c64b16eb4e2bc10312dba48eaaf4189c1abc3c97a0d26144f8e9aee72: Status 404 returned error can't find the container with id 165b1b9c64b16eb4e2bc10312dba48eaaf4189c1abc3c97a0d26144f8e9aee72
Jul 06 18:17:46.561181 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.536933    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-sl8sj"
Jul 06 18:17:46.561181 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:46.545631    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda420108c_26d7_46d2_a3d5_3b354aa01af0.slice/crio-ce90e50a4e2e604b9042f6569865df1aafd62a5303de2fd245e912b6a46b0557 WatchSource:0}: Error finding container ce90e50a4e2e604b9042f6569865df1aafd62a5303de2fd245e912b6a46b0557: Status 404 returned error can't find the container with id ce90e50a4e2e604b9042f6569865df1aafd62a5303de2fd245e912b6a46b0557
Jul 06 18:17:46.561181 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.552879    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-zk2x7"
Jul 06 18:17:46.810427 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:46.564886    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc5411d9d_cd87_44a4_a2b8_99acc89e5a2f.slice/crio-55986a53ed9f21215938174986dc7b22752fbb0cb332a6071a1e0670a39e74ce WatchSource:0}: Error finding container 55986a53ed9f21215938174986dc7b22752fbb0cb332a6071a1e0670a39e74ce: Status 404 returned error can't find the container with id 55986a53ed9f21215938174986dc7b22752fbb0cb332a6071a1e0670a39e74ce
Jul 06 18:17:46.810427 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.573568    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-vz7bl"
Jul 06 18:17:46.810427 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:46.580383    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf914b85_02d3_423a_aed5_e96b26e6c1c1.slice/crio-250134175e43405a9311e66acf2d9c5469dfd4c6ddfa74d009d316e46c02c468 WatchSource:0}: Error finding container 250134175e43405a9311e66acf2d9c5469dfd4c6ddfa74d009d316e46c02c468: Status 404 returned error can't find the container with id 250134175e43405a9311e66acf2d9c5469dfd4c6ddfa74d009d316e46c02c468
Jul 06 18:17:46.810427 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.583626    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ztxm5"
Jul 06 18:17:46.810427 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.589308    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-b7jm9"
Jul 06 18:17:46.810427 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:46.590281    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0474b033_5051_4df3_8135_4b05741835ac.slice/crio-13d7dd2d8da884aa30fa1f4ba859dbcdba7bc1ca927573efbf45988d6e6533eb WatchSource:0}: Error finding container 13d7dd2d8da884aa30fa1f4ba859dbcdba7bc1ca927573efbf45988d6e6533eb: Status 404 returned error can't find the container with id 13d7dd2d8da884aa30fa1f4ba859dbcdba7bc1ca927573efbf45988d6e6533eb
Jul 06 18:17:46.810427 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.596343    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-x7rd5"
Jul 06 18:17:46.810427 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:46.596917    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf117fdd5_8a61_484e_8ebc_668f89723813.slice/crio-86ba20f1746e5cd5ab63ad6368d4e313f107aa02d858c91888ac07cdb2461a22 WatchSource:0}: Error finding container 86ba20f1746e5cd5ab63ad6368d4e313f107aa02d858c91888ac07cdb2461a22: Status 404 returned error can't find the container with id 86ba20f1746e5cd5ab63ad6368d4e313f107aa02d858c91888ac07cdb2461a22
Jul 06 18:17:46.810427 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:46.602764    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd27a6ce1_0910_4c08_959a_550b8d468219.slice/crio-e6ec58be05ea0a0ffe57b47e6471019dcb34e3eb12dc37e87f38a24e1c6e3b7f WatchSource:0}: Error finding container e6ec58be05ea0a0ffe57b47e6471019dcb34e3eb12dc37e87f38a24e1c6e3b7f: Status 404 returned error can't find the container with id e6ec58be05ea0a0ffe57b47e6471019dcb34e3eb12dc37e87f38a24e1c6e3b7f
Jul 06 18:17:46.810427 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.603493    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-rttvb"
Jul 06 18:17:46.810427 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.609037    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-kcr4l"
Jul 06 18:17:46.810427 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:46.612102    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b707150_01d0_47c3_b61e_a9a6f33465fc.slice/crio-3e8f6edb1e90a5b1d63037218aa44793dc79e7506bdc7524e922aac851ed1caf WatchSource:0}: Error finding container 3e8f6edb1e90a5b1d63037218aa44793dc79e7506bdc7524e922aac851ed1caf: Status 404 returned error can't find the container with id 3e8f6edb1e90a5b1d63037218aa44793dc79e7506bdc7524e922aac851ed1caf
Jul 06 18:17:46.810427 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:17:46.615554    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2ef9254_5ac3_45db_99a0_b5aff2a25d58.slice/crio-84933b0521f8ed339a0cebc85bb94d3a05dcb0fcb44933231a07a3cc28e6bc37 WatchSource:0}: Error finding container 84933b0521f8ed339a0cebc85bb94d3a05dcb0fcb44933231a07a3cc28e6bc37: Status 404 returned error can't find the container with id 84933b0521f8ed339a0cebc85bb94d3a05dcb0fcb44933231a07a3cc28e6bc37
Jul 06 18:17:46.814690 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.814293    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-knzwx\" (UniqueName: \"kubernetes.io/projected/4e2f69c5-ef58-430a-8b74-eb53ce230c7e-kube-api-access-knzwx\") pod \"network-check-target-fp8fn\" (UID: \"4e2f69c5-ef58-430a-8b74-eb53ce230c7e\") " pod="openshift-network-diagnostics/network-check-target-fp8fn"
Jul 06 18:17:46.814690 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:17:46.814453    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 06 18:17:46.814690 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:17:46.814471    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 06 18:17:46.814690 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:17:46.814482    2580 projected.go:196] Error preparing data for projected volume kube-api-access-knzwx for pod openshift-network-diagnostics/network-check-target-fp8fn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 18:17:46.814690 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:17:46.814531    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4e2f69c5-ef58-430a-8b74-eb53ce230c7e-kube-api-access-knzwx podName:4e2f69c5-ef58-430a-8b74-eb53ce230c7e nodeName:}" failed. No retries permitted until 2026-07-06 18:17:47.814517732 +0000 UTC m=+2.971916196 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-knzwx" (UniqueName: "kubernetes.io/projected/4e2f69c5-ef58-430a-8b74-eb53ce230c7e-kube-api-access-knzwx") pod "network-check-target-fp8fn" (UID: "4e2f69c5-ef58-430a-8b74-eb53ce230c7e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 18:17:46.965707 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.915034    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3b6b323e-eca7-4c68-87a7-0d48a9114a2c-original-pull-secret\") pod \"global-pull-secret-syncer-xzqch\" (UID: \"3b6b323e-eca7-4c68-87a7-0d48a9114a2c\") " pod="kube-system/global-pull-secret-syncer-xzqch"
Jul 06 18:17:46.965707 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:46.915091    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f22387dc-2c30-4f80-8a80-3f6420c1c858-metrics-certs\") pod \"network-metrics-daemon-xjbzr\" (UID: \"f22387dc-2c30-4f80-8a80-3f6420c1c858\") " pod="openshift-multus/network-metrics-daemon-xjbzr"
Jul 06 18:17:46.965707 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:17:46.915162    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 06 18:17:46.965707 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:17:46.915221    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 18:17:46.965707 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:17:46.915243    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b6b323e-eca7-4c68-87a7-0d48a9114a2c-original-pull-secret podName:3b6b323e-eca7-4c68-87a7-0d48a9114a2c nodeName:}" failed. No retries permitted until 2026-07-06 18:17:47.915222645 +0000 UTC m=+3.072621112 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3b6b323e-eca7-4c68-87a7-0d48a9114a2c-original-pull-secret") pod "global-pull-secret-syncer-xzqch" (UID: "3b6b323e-eca7-4c68-87a7-0d48a9114a2c") : object "kube-system"/"original-pull-secret" not registered
Jul 06 18:17:46.965707 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:17:46.915269    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f22387dc-2c30-4f80-8a80-3f6420c1c858-metrics-certs podName:f22387dc-2c30-4f80-8a80-3f6420c1c858 nodeName:}" failed. No retries permitted until 2026-07-06 18:17:47.915253605 +0000 UTC m=+3.072652086 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f22387dc-2c30-4f80-8a80-3f6420c1c858-metrics-certs") pod "network-metrics-daemon-xjbzr" (UID: "f22387dc-2c30-4f80-8a80-3f6420c1c858") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 18:17:47.235879 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:47.235777    2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-05 18:12:46 +0000 UTC" deadline="2028-04-14 10:42:56.245767356 +0000 UTC"
Jul 06 18:17:47.235879 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:47.235824    2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15544h25m9.009948569s"
Jul 06 18:17:47.318795 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:47.318131    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xjbzr"
Jul 06 18:17:47.318795 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:17:47.318269    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xjbzr" podUID="f22387dc-2c30-4f80-8a80-3f6420c1c858"
Jul 06 18:17:47.320142 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:47.319703    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fp8fn"
Jul 06 18:17:47.320142 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:17:47.319803    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fp8fn" podUID="4e2f69c5-ef58-430a-8b74-eb53ce230c7e"
Jul 06 18:17:47.351311 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:47.351229    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-kcr4l" event={"ID":"c2ef9254-5ac3-45db-99a0-b5aff2a25d58","Type":"ContainerStarted","Data":"84933b0521f8ed339a0cebc85bb94d3a05dcb0fcb44933231a07a3cc28e6bc37"}
Jul 06 18:17:47.356747 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:47.356681    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-vz7bl" event={"ID":"af914b85-02d3-423a-aed5-e96b26e6c1c1","Type":"ContainerStarted","Data":"250134175e43405a9311e66acf2d9c5469dfd4c6ddfa74d009d316e46c02c468"}
Jul 06 18:17:47.373607 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:47.373531    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-sl8sj" event={"ID":"a420108c-26d7-46d2-a3d5-3b354aa01af0","Type":"ContainerStarted","Data":"ce90e50a4e2e604b9042f6569865df1aafd62a5303de2fd245e912b6a46b0557"}
Jul 06 18:17:47.390298 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:47.390176    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9h479" event={"ID":"5cfed2e4-8993-42f8-bc84-f9399b287729","Type":"ContainerStarted","Data":"165b1b9c64b16eb4e2bc10312dba48eaaf4189c1abc3c97a0d26144f8e9aee72"}
Jul 06 18:17:47.409827 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:47.409789    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-rttvb" event={"ID":"7b707150-01d0-47c3-b61e-a9a6f33465fc","Type":"ContainerStarted","Data":"3e8f6edb1e90a5b1d63037218aa44793dc79e7506bdc7524e922aac851ed1caf"}
Jul 06 18:17:47.419241 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:47.419142    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-x7rd5" event={"ID":"d27a6ce1-0910-4c08-959a-550b8d468219","Type":"ContainerStarted","Data":"e6ec58be05ea0a0ffe57b47e6471019dcb34e3eb12dc37e87f38a24e1c6e3b7f"}
Jul 06 18:17:47.428108 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:47.427999    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-b7jm9" event={"ID":"f117fdd5-8a61-484e-8ebc-668f89723813","Type":"ContainerStarted","Data":"86ba20f1746e5cd5ab63ad6368d4e313f107aa02d858c91888ac07cdb2461a22"}
Jul 06 18:17:47.433073 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:47.433032    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ztxm5" event={"ID":"0474b033-5051-4df3-8135-4b05741835ac","Type":"ContainerStarted","Data":"13d7dd2d8da884aa30fa1f4ba859dbcdba7bc1ca927573efbf45988d6e6533eb"}
Jul 06 18:17:47.445200 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:47.445101    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-zk2x7" event={"ID":"c5411d9d-cd87-44a4-a2b8-99acc89e5a2f","Type":"ContainerStarted","Data":"55986a53ed9f21215938174986dc7b22752fbb0cb332a6071a1e0670a39e74ce"}
Jul 06 18:17:47.598666 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:47.598394    2580 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 06 18:17:47.821613 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:47.821574    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-knzwx\" (UniqueName: \"kubernetes.io/projected/4e2f69c5-ef58-430a-8b74-eb53ce230c7e-kube-api-access-knzwx\") pod \"network-check-target-fp8fn\" (UID: \"4e2f69c5-ef58-430a-8b74-eb53ce230c7e\") " pod="openshift-network-diagnostics/network-check-target-fp8fn"
Jul 06 18:17:47.821795 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:17:47.821778    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 06 18:17:47.821856 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:17:47.821806    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 06 18:17:47.821856 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:17:47.821819    2580 projected.go:196] Error preparing data for projected volume kube-api-access-knzwx for pod openshift-network-diagnostics/network-check-target-fp8fn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 18:17:47.821963 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:17:47.821881    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4e2f69c5-ef58-430a-8b74-eb53ce230c7e-kube-api-access-knzwx podName:4e2f69c5-ef58-430a-8b74-eb53ce230c7e nodeName:}" failed. No retries permitted until 2026-07-06 18:17:49.821862079 +0000 UTC m=+4.979260563 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-knzwx" (UniqueName: "kubernetes.io/projected/4e2f69c5-ef58-430a-8b74-eb53ce230c7e-kube-api-access-knzwx") pod "network-check-target-fp8fn" (UID: "4e2f69c5-ef58-430a-8b74-eb53ce230c7e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 18:17:47.922807 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:47.922299    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3b6b323e-eca7-4c68-87a7-0d48a9114a2c-original-pull-secret\") pod \"global-pull-secret-syncer-xzqch\" (UID: \"3b6b323e-eca7-4c68-87a7-0d48a9114a2c\") " pod="kube-system/global-pull-secret-syncer-xzqch"
Jul 06 18:17:47.922807 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:47.922350    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f22387dc-2c30-4f80-8a80-3f6420c1c858-metrics-certs\") pod \"network-metrics-daemon-xjbzr\" (UID: \"f22387dc-2c30-4f80-8a80-3f6420c1c858\") " pod="openshift-multus/network-metrics-daemon-xjbzr"
Jul 06 18:17:47.922807 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:17:47.922481    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 18:17:47.922807 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:17:47.922546    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f22387dc-2c30-4f80-8a80-3f6420c1c858-metrics-certs podName:f22387dc-2c30-4f80-8a80-3f6420c1c858 nodeName:}" failed. No retries permitted until 2026-07-06 18:17:49.92252698 +0000 UTC m=+5.079925466 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f22387dc-2c30-4f80-8a80-3f6420c1c858-metrics-certs") pod "network-metrics-daemon-xjbzr" (UID: "f22387dc-2c30-4f80-8a80-3f6420c1c858") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 18:17:47.922807 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:17:47.922576    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 06 18:17:47.922807 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:17:47.922644    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b6b323e-eca7-4c68-87a7-0d48a9114a2c-original-pull-secret podName:3b6b323e-eca7-4c68-87a7-0d48a9114a2c nodeName:}" failed. No retries permitted until 2026-07-06 18:17:49.922625468 +0000 UTC m=+5.080023935 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3b6b323e-eca7-4c68-87a7-0d48a9114a2c-original-pull-secret") pod "global-pull-secret-syncer-xzqch" (UID: "3b6b323e-eca7-4c68-87a7-0d48a9114a2c") : object "kube-system"/"original-pull-secret" not registered
Jul 06 18:17:47.997681 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:47.997651    2580 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 06 18:17:48.236633 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:48.236523    2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-05 18:12:46 +0000 UTC" deadline="2027-12-22 12:48:14.060566764 +0000 UTC"
Jul 06 18:17:48.236633 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:48.236583    2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12810h30m25.823986963s"
Jul 06 18:17:48.317420 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:48.317390    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xzqch"
Jul 06 18:17:48.317595 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:17:48.317526    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xzqch" podUID="3b6b323e-eca7-4c68-87a7-0d48a9114a2c"
Jul 06 18:17:49.317665 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:49.317579    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xjbzr"
Jul 06 18:17:49.318095 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:17:49.317726    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xjbzr" podUID="f22387dc-2c30-4f80-8a80-3f6420c1c858"
Jul 06 18:17:49.320071 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:49.317533    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fp8fn"
Jul 06 18:17:49.320071 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:17:49.320001    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fp8fn" podUID="4e2f69c5-ef58-430a-8b74-eb53ce230c7e"
Jul 06 18:17:49.839502 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:49.839463    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-knzwx\" (UniqueName: \"kubernetes.io/projected/4e2f69c5-ef58-430a-8b74-eb53ce230c7e-kube-api-access-knzwx\") pod \"network-check-target-fp8fn\" (UID: \"4e2f69c5-ef58-430a-8b74-eb53ce230c7e\") " pod="openshift-network-diagnostics/network-check-target-fp8fn"
Jul 06 18:17:49.839692 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:17:49.839641    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 06 18:17:49.839692 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:17:49.839667    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 06 18:17:49.839692 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:17:49.839682    2580 projected.go:196] Error preparing data for projected volume kube-api-access-knzwx for pod openshift-network-diagnostics/network-check-target-fp8fn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 18:17:49.839966 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:17:49.839740    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4e2f69c5-ef58-430a-8b74-eb53ce230c7e-kube-api-access-knzwx podName:4e2f69c5-ef58-430a-8b74-eb53ce230c7e nodeName:}" failed. No retries permitted until 2026-07-06 18:17:53.839720827 +0000 UTC m=+8.997119310 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-knzwx" (UniqueName: "kubernetes.io/projected/4e2f69c5-ef58-430a-8b74-eb53ce230c7e-kube-api-access-knzwx") pod "network-check-target-fp8fn" (UID: "4e2f69c5-ef58-430a-8b74-eb53ce230c7e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 18:17:49.941422 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:49.940841    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3b6b323e-eca7-4c68-87a7-0d48a9114a2c-original-pull-secret\") pod \"global-pull-secret-syncer-xzqch\" (UID: \"3b6b323e-eca7-4c68-87a7-0d48a9114a2c\") " pod="kube-system/global-pull-secret-syncer-xzqch"
Jul 06 18:17:49.941422 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:49.940889    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f22387dc-2c30-4f80-8a80-3f6420c1c858-metrics-certs\") pod \"network-metrics-daemon-xjbzr\" (UID: \"f22387dc-2c30-4f80-8a80-3f6420c1c858\") " pod="openshift-multus/network-metrics-daemon-xjbzr"
Jul 06 18:17:49.941422 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:17:49.941009    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 18:17:49.941422 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:17:49.941087    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f22387dc-2c30-4f80-8a80-3f6420c1c858-metrics-certs podName:f22387dc-2c30-4f80-8a80-3f6420c1c858 nodeName:}" failed. No retries permitted until 2026-07-06 18:17:53.941068942 +0000 UTC m=+9.098467411 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f22387dc-2c30-4f80-8a80-3f6420c1c858-metrics-certs") pod "network-metrics-daemon-xjbzr" (UID: "f22387dc-2c30-4f80-8a80-3f6420c1c858") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 18:17:49.941422 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:17:49.941293    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 06 18:17:49.941422 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:17:49.941355    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b6b323e-eca7-4c68-87a7-0d48a9114a2c-original-pull-secret podName:3b6b323e-eca7-4c68-87a7-0d48a9114a2c nodeName:}" failed. No retries permitted until 2026-07-06 18:17:53.941337917 +0000 UTC m=+9.098736393 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3b6b323e-eca7-4c68-87a7-0d48a9114a2c-original-pull-secret") pod "global-pull-secret-syncer-xzqch" (UID: "3b6b323e-eca7-4c68-87a7-0d48a9114a2c") : object "kube-system"/"original-pull-secret" not registered
Jul 06 18:17:50.317126 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:50.317049    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xzqch"
Jul 06 18:17:50.317314 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:17:50.317207    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xzqch" podUID="3b6b323e-eca7-4c68-87a7-0d48a9114a2c"
Jul 06 18:17:51.318083 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:51.318037    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fp8fn"
Jul 06 18:17:51.318565 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:17:51.318168    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fp8fn" podUID="4e2f69c5-ef58-430a-8b74-eb53ce230c7e"
Jul 06 18:17:51.318565 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:51.318543    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xjbzr"
Jul 06 18:17:51.318676 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:17:51.318652    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xjbzr" podUID="f22387dc-2c30-4f80-8a80-3f6420c1c858"
Jul 06 18:17:52.317158 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:52.317068    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xzqch"
Jul 06 18:17:52.317350 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:17:52.317206    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xzqch" podUID="3b6b323e-eca7-4c68-87a7-0d48a9114a2c"
Jul 06 18:17:53.317673 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:53.317557    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fp8fn"
Jul 06 18:17:53.318059 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:17:53.317693    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fp8fn" podUID="4e2f69c5-ef58-430a-8b74-eb53ce230c7e"
Jul 06 18:17:53.318059 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:53.317978    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xjbzr"
Jul 06 18:17:53.318184 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:17:53.318104    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xjbzr" podUID="f22387dc-2c30-4f80-8a80-3f6420c1c858"
Jul 06 18:17:53.869841 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:53.869783    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-knzwx\" (UniqueName: \"kubernetes.io/projected/4e2f69c5-ef58-430a-8b74-eb53ce230c7e-kube-api-access-knzwx\") pod \"network-check-target-fp8fn\" (UID: \"4e2f69c5-ef58-430a-8b74-eb53ce230c7e\") " pod="openshift-network-diagnostics/network-check-target-fp8fn"
Jul 06 18:17:53.870040 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:17:53.869919    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 06 18:17:53.870040 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:17:53.869941    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 06 18:17:53.870040 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:17:53.869953    2580 projected.go:196] Error preparing data for projected volume kube-api-access-knzwx for pod openshift-network-diagnostics/network-check-target-fp8fn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 18:17:53.870040 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:17:53.870027    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4e2f69c5-ef58-430a-8b74-eb53ce230c7e-kube-api-access-knzwx podName:4e2f69c5-ef58-430a-8b74-eb53ce230c7e nodeName:}" failed. No retries permitted until 2026-07-06 18:18:01.869990098 +0000 UTC m=+17.027388576 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-knzwx" (UniqueName: "kubernetes.io/projected/4e2f69c5-ef58-430a-8b74-eb53ce230c7e-kube-api-access-knzwx") pod "network-check-target-fp8fn" (UID: "4e2f69c5-ef58-430a-8b74-eb53ce230c7e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 18:17:53.970740 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:53.970704    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3b6b323e-eca7-4c68-87a7-0d48a9114a2c-original-pull-secret\") pod \"global-pull-secret-syncer-xzqch\" (UID: \"3b6b323e-eca7-4c68-87a7-0d48a9114a2c\") " pod="kube-system/global-pull-secret-syncer-xzqch"
Jul 06 18:17:53.970740 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:53.970741    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f22387dc-2c30-4f80-8a80-3f6420c1c858-metrics-certs\") pod \"network-metrics-daemon-xjbzr\" (UID: \"f22387dc-2c30-4f80-8a80-3f6420c1c858\") " pod="openshift-multus/network-metrics-daemon-xjbzr"
Jul 06 18:17:53.970961 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:17:53.970865    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 18:17:53.970961 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:17:53.970919    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f22387dc-2c30-4f80-8a80-3f6420c1c858-metrics-certs podName:f22387dc-2c30-4f80-8a80-3f6420c1c858 nodeName:}" failed. No retries permitted until 2026-07-06 18:18:01.970906049 +0000 UTC m=+17.128304514 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f22387dc-2c30-4f80-8a80-3f6420c1c858-metrics-certs") pod "network-metrics-daemon-xjbzr" (UID: "f22387dc-2c30-4f80-8a80-3f6420c1c858") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 18:17:53.970961 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:17:53.970865    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 06 18:17:53.971141 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:17:53.971025    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b6b323e-eca7-4c68-87a7-0d48a9114a2c-original-pull-secret podName:3b6b323e-eca7-4c68-87a7-0d48a9114a2c nodeName:}" failed. No retries permitted until 2026-07-06 18:18:01.970993461 +0000 UTC m=+17.128391930 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3b6b323e-eca7-4c68-87a7-0d48a9114a2c-original-pull-secret") pod "global-pull-secret-syncer-xzqch" (UID: "3b6b323e-eca7-4c68-87a7-0d48a9114a2c") : object "kube-system"/"original-pull-secret" not registered
Jul 06 18:17:54.317499 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:54.317416    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xzqch"
Jul 06 18:17:54.317650 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:17:54.317548    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xzqch" podUID="3b6b323e-eca7-4c68-87a7-0d48a9114a2c"
Jul 06 18:17:55.318132 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:55.318095    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fp8fn"
Jul 06 18:17:55.318547 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:17:55.318217    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fp8fn" podUID="4e2f69c5-ef58-430a-8b74-eb53ce230c7e"
Jul 06 18:17:55.318681 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:55.318662    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xjbzr"
Jul 06 18:17:55.318816 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:17:55.318793    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xjbzr" podUID="f22387dc-2c30-4f80-8a80-3f6420c1c858"
Jul 06 18:17:56.317191 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:56.317149    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xzqch"
Jul 06 18:17:56.317440 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:17:56.317414    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xzqch" podUID="3b6b323e-eca7-4c68-87a7-0d48a9114a2c"
Jul 06 18:17:57.317410 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:57.317371    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fp8fn"
Jul 06 18:17:57.317852 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:17:57.317511    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fp8fn" podUID="4e2f69c5-ef58-430a-8b74-eb53ce230c7e"
Jul 06 18:17:57.317852 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:57.317809    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xjbzr"
Jul 06 18:17:57.317977 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:17:57.317933    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xjbzr" podUID="f22387dc-2c30-4f80-8a80-3f6420c1c858"
Jul 06 18:17:58.317500 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:58.317466    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xzqch"
Jul 06 18:17:58.317952 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:17:58.317591    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xzqch" podUID="3b6b323e-eca7-4c68-87a7-0d48a9114a2c"
Jul 06 18:17:59.319917 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:59.319887    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xjbzr"
Jul 06 18:17:59.320366 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:17:59.320034    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xjbzr" podUID="f22387dc-2c30-4f80-8a80-3f6420c1c858"
Jul 06 18:17:59.320366 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:17:59.320044    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fp8fn"
Jul 06 18:17:59.320366 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:17:59.320110    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fp8fn" podUID="4e2f69c5-ef58-430a-8b74-eb53ce230c7e"
Jul 06 18:18:00.317735 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:00.317702    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xzqch"
Jul 06 18:18:00.317904 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:18:00.317838    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xzqch" podUID="3b6b323e-eca7-4c68-87a7-0d48a9114a2c"
Jul 06 18:18:01.318057 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:01.317848    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xjbzr"
Jul 06 18:18:01.318479 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:01.317845    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fp8fn"
Jul 06 18:18:01.318479 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:18:01.318166    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xjbzr" podUID="f22387dc-2c30-4f80-8a80-3f6420c1c858"
Jul 06 18:18:01.318479 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:18:01.318215    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fp8fn" podUID="4e2f69c5-ef58-430a-8b74-eb53ce230c7e"
Jul 06 18:18:01.929780 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:01.929719    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-knzwx\" (UniqueName: \"kubernetes.io/projected/4e2f69c5-ef58-430a-8b74-eb53ce230c7e-kube-api-access-knzwx\") pod \"network-check-target-fp8fn\" (UID: \"4e2f69c5-ef58-430a-8b74-eb53ce230c7e\") " pod="openshift-network-diagnostics/network-check-target-fp8fn"
Jul 06 18:18:01.930007 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:18:01.929888    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 06 18:18:01.930007 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:18:01.929914    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 06 18:18:01.930007 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:18:01.929926    2580 projected.go:196] Error preparing data for projected volume kube-api-access-knzwx for pod openshift-network-diagnostics/network-check-target-fp8fn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 18:18:01.930007 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:18:01.929979    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4e2f69c5-ef58-430a-8b74-eb53ce230c7e-kube-api-access-knzwx podName:4e2f69c5-ef58-430a-8b74-eb53ce230c7e nodeName:}" failed. No retries permitted until 2026-07-06 18:18:17.929965042 +0000 UTC m=+33.087363523 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-knzwx" (UniqueName: "kubernetes.io/projected/4e2f69c5-ef58-430a-8b74-eb53ce230c7e-kube-api-access-knzwx") pod "network-check-target-fp8fn" (UID: "4e2f69c5-ef58-430a-8b74-eb53ce230c7e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 18:18:02.030130 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:02.030091    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3b6b323e-eca7-4c68-87a7-0d48a9114a2c-original-pull-secret\") pod \"global-pull-secret-syncer-xzqch\" (UID: \"3b6b323e-eca7-4c68-87a7-0d48a9114a2c\") " pod="kube-system/global-pull-secret-syncer-xzqch"
Jul 06 18:18:02.030130 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:02.030139    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f22387dc-2c30-4f80-8a80-3f6420c1c858-metrics-certs\") pod \"network-metrics-daemon-xjbzr\" (UID: \"f22387dc-2c30-4f80-8a80-3f6420c1c858\") " pod="openshift-multus/network-metrics-daemon-xjbzr"
Jul 06 18:18:02.030352 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:18:02.030222    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 06 18:18:02.030352 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:18:02.030255    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 18:18:02.030352 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:18:02.030291    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b6b323e-eca7-4c68-87a7-0d48a9114a2c-original-pull-secret podName:3b6b323e-eca7-4c68-87a7-0d48a9114a2c nodeName:}" failed. No retries permitted until 2026-07-06 18:18:18.030272289 +0000 UTC m=+33.187670774 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3b6b323e-eca7-4c68-87a7-0d48a9114a2c-original-pull-secret") pod "global-pull-secret-syncer-xzqch" (UID: "3b6b323e-eca7-4c68-87a7-0d48a9114a2c") : object "kube-system"/"original-pull-secret" not registered
Jul 06 18:18:02.030352 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:18:02.030312    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f22387dc-2c30-4f80-8a80-3f6420c1c858-metrics-certs podName:f22387dc-2c30-4f80-8a80-3f6420c1c858 nodeName:}" failed. No retries permitted until 2026-07-06 18:18:18.030301892 +0000 UTC m=+33.187700356 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f22387dc-2c30-4f80-8a80-3f6420c1c858-metrics-certs") pod "network-metrics-daemon-xjbzr" (UID: "f22387dc-2c30-4f80-8a80-3f6420c1c858") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 18:18:02.317855 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:02.317763    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xzqch"
Jul 06 18:18:02.318035 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:18:02.317892    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xzqch" podUID="3b6b323e-eca7-4c68-87a7-0d48a9114a2c"
Jul 06 18:18:03.317442 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:03.317407    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fp8fn"
Jul 06 18:18:03.317899 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:18:03.317536    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fp8fn" podUID="4e2f69c5-ef58-430a-8b74-eb53ce230c7e"
Jul 06 18:18:03.317899 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:03.317594    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xjbzr"
Jul 06 18:18:03.317899 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:18:03.317729    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xjbzr" podUID="f22387dc-2c30-4f80-8a80-3f6420c1c858"
Jul 06 18:18:04.317484 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:04.317458    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xzqch"
Jul 06 18:18:04.317904 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:18:04.317564    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xzqch" podUID="3b6b323e-eca7-4c68-87a7-0d48a9114a2c"
Jul 06 18:18:05.318417 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:05.318379    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fp8fn"
Jul 06 18:18:05.318839 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:18:05.318486    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fp8fn" podUID="4e2f69c5-ef58-430a-8b74-eb53ce230c7e"
Jul 06 18:18:05.318839 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:05.318575    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xjbzr"
Jul 06 18:18:05.318839 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:18:05.318705    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xjbzr" podUID="f22387dc-2c30-4f80-8a80-3f6420c1c858"
Jul 06 18:18:06.317559 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:06.317538    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xzqch"
Jul 06 18:18:06.317648 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:18:06.317630    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xzqch" podUID="3b6b323e-eca7-4c68-87a7-0d48a9114a2c"
Jul 06 18:18:07.318009 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:07.317796    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fp8fn"
Jul 06 18:18:07.318653 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:07.317851    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xjbzr"
Jul 06 18:18:07.318653 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:18:07.318074    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fp8fn" podUID="4e2f69c5-ef58-430a-8b74-eb53ce230c7e"
Jul 06 18:18:07.318653 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:18:07.318115    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xjbzr" podUID="f22387dc-2c30-4f80-8a80-3f6420c1c858"
Jul 06 18:18:07.514098 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:07.514056    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-rttvb" event={"ID":"7b707150-01d0-47c3-b61e-a9a6f33465fc","Type":"ContainerStarted","Data":"6081743031acb12e5ec786185cb654785cfaf389ad0ed1bbdba02898f3da114b"}
Jul 06 18:18:07.515425 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:07.515401    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-x7rd5" event={"ID":"d27a6ce1-0910-4c08-959a-550b8d468219","Type":"ContainerStarted","Data":"7facd0dc2f18baa7f9b52a5e44c7abb389f006657a48eb29423e56ca6fc7e7d9"}
Jul 06 18:18:07.516740 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:07.516723    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-b7jm9" event={"ID":"f117fdd5-8a61-484e-8ebc-668f89723813","Type":"ContainerStarted","Data":"5e2e1cf18f93a7d0f0a7cff97bf50680496dfba44d268e28218b229d122d32a3"}
Jul 06 18:18:07.518058 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:07.518040    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ztxm5" event={"ID":"0474b033-5051-4df3-8135-4b05741835ac","Type":"ContainerStarted","Data":"7a5602393b12d4721818f0f4fd5f58ef53e8b416d235aeac9d73347a75fdb3fd"}
Jul 06 18:18:07.519355 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:07.519330    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-zk2x7" event={"ID":"c5411d9d-cd87-44a4-a2b8-99acc89e5a2f","Type":"ContainerStarted","Data":"cba033cf39c96e8cec80c6f25f705c69cbcc8fc84c48bac73a519d599b6277fe"}
Jul 06 18:18:07.520648 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:07.520629    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-kcr4l" event={"ID":"c2ef9254-5ac3-45db-99a0-b5aff2a25d58","Type":"ContainerStarted","Data":"5f7f393263e7c7df2d2ab53ff6b52adf184314ac19b1cbbb30d076d06708938a"}
Jul 06 18:18:07.521933 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:07.521908    2580 generic.go:356] "Generic (PLEG): container finished" podID="a420108c-26d7-46d2-a3d5-3b354aa01af0" containerID="00c331c979787c3c8b703292050f7bfdd884f43ed82f4b4ea5f72330407272f6" exitCode=0
Jul 06 18:18:07.522010 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:07.521980    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-sl8sj" event={"ID":"a420108c-26d7-46d2-a3d5-3b354aa01af0","Type":"ContainerDied","Data":"00c331c979787c3c8b703292050f7bfdd884f43ed82f4b4ea5f72330407272f6"}
Jul 06 18:18:07.524908 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:07.524884    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9h479" event={"ID":"5cfed2e4-8993-42f8-bc84-f9399b287729","Type":"ContainerStarted","Data":"7cd973d566d1f58e1383515c623612658ab3635b78b5912e565e19b84d52a13a"}
Jul 06 18:18:07.524988 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:07.524940    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9h479" event={"ID":"5cfed2e4-8993-42f8-bc84-f9399b287729","Type":"ContainerStarted","Data":"5f953d5c65bd9747eac3e7c90a810a317d80b388e3a74ff637573475ab06e2e3"}
Jul 06 18:18:07.524988 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:07.524980    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9h479" event={"ID":"5cfed2e4-8993-42f8-bc84-f9399b287729","Type":"ContainerStarted","Data":"79680e0c8df2d71a4be8b98da1d3c86df5420dadb0e32fe43e6164bd055c0543"}
Jul 06 18:18:07.525089 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:07.525034    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9h479" event={"ID":"5cfed2e4-8993-42f8-bc84-f9399b287729","Type":"ContainerStarted","Data":"b43c8bbe8a957a540f545345ba5bbf732eecf4da0d3129a268ff997a5635f146"}
Jul 06 18:18:07.525089 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:07.525070    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9h479" event={"ID":"5cfed2e4-8993-42f8-bc84-f9399b287729","Type":"ContainerStarted","Data":"168540eee994c547e594e03f76df2a98c5cb54099c43a38bdcb67aaf023fb1b1"}
Jul 06 18:18:07.525193 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:07.525102    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9h479" event={"ID":"5cfed2e4-8993-42f8-bc84-f9399b287729","Type":"ContainerStarted","Data":"94162fdd622ce4af948ebdc8fb9bd7916c9ca166a3598433c63b1eabe5e131c6"}
Jul 06 18:18:07.526295 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:07.526271    2580 generic.go:356] "Generic (PLEG): container finished" podID="6eab86221f4cc923c73e223abc50211e" containerID="a42fc9716985230824bff37bea089486478082166fe2f465df93652611f1137d" exitCode=0
Jul 06 18:18:07.526363 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:07.526342    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-113.ec2.internal" event={"ID":"6eab86221f4cc923c73e223abc50211e","Type":"ContainerDied","Data":"a42fc9716985230824bff37bea089486478082166fe2f465df93652611f1137d"}
Jul 06 18:18:07.527727 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:07.527709    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-113.ec2.internal" event={"ID":"66d40220305fe03479bbe46175f2cd79","Type":"ContainerStarted","Data":"9d9fbad3f4445356d31ff2283ac6f8ca1fa97630554b2b3be278b38e11706e69"}
Jul 06 18:18:07.547138 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:07.547096    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-rttvb" podStartSLOduration=2.823896255 podStartE2EDuration="22.547082272s" totalImagesPullingTime="19.723186017s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 18:17:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:18:07.546961308 +0000 UTC m=+22.704359795" watchObservedRunningTime="2026-07-06 18:18:07.547082272 +0000 UTC m=+22.704480757"
Jul 06 18:18:07.598849 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:07.598795    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-kcr4l" podStartSLOduration=2.926168024 podStartE2EDuration="22.598773541s" totalImagesPullingTime="19.672605517s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 18:17:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:18:07.598771457 +0000 UTC m=+22.756169957" watchObservedRunningTime="2026-07-06 18:18:07.598773541 +0000 UTC m=+22.756172031"
Jul 06 18:18:07.599065 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:07.599040    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-x7rd5" podStartSLOduration=2.869220238 podStartE2EDuration="22.599030849s" totalImagesPullingTime="19.729810611s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 18:17:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:18:07.57896133 +0000 UTC m=+22.736359816" watchObservedRunningTime="2026-07-06 18:18:07.599030849 +0000 UTC m=+22.756429333"
Jul 06 18:18:07.626096 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:07.626044    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-113.ec2.internal" podStartSLOduration=21.626030536000002 podStartE2EDuration="21.626030536s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 18:17:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:18:07.625870402 +0000 UTC m=+22.783268888" watchObservedRunningTime="2026-07-06 18:18:07.626030536 +0000 UTC m=+22.783429023"
Jul 06 18:18:07.707392 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:07.707130    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-zk2x7" podStartSLOduration=2.766261756 podStartE2EDuration="22.70710939s" totalImagesPullingTime="19.940847634s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 18:17:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:18:07.687198226 +0000 UTC m=+22.844596716" watchObservedRunningTime="2026-07-06 18:18:07.70710939 +0000 UTC m=+22.864507880"
Jul 06 18:18:07.708129 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:07.708089    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-b7jm9" podStartSLOduration=2.972753194 podStartE2EDuration="22.708071611s" totalImagesPullingTime="19.735318417s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:17:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:18:07.707474402 +0000 UTC m=+22.864872882" watchObservedRunningTime="2026-07-06 18:18:07.708071611 +0000 UTC m=+22.865470098"
Jul 06 18:18:08.187396 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:08.187370    2580 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 06 18:18:08.269564 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:08.269412    2580 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-06T18:18:08.187390195Z","UUID":"9515efc0-a58c-4871-a359-ed9b06c6f8dc","Handler":null,"Name":"","Endpoint":""}
Jul 06 18:18:08.271126 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:08.271100    2580 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 06 18:18:08.271258 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:08.271146    2580 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 06 18:18:08.317165 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:08.317129    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xzqch"
Jul 06 18:18:08.317338 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:18:08.317261    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xzqch" podUID="3b6b323e-eca7-4c68-87a7-0d48a9114a2c"
Jul 06 18:18:08.532691 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:08.532591    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-113.ec2.internal" event={"ID":"6eab86221f4cc923c73e223abc50211e","Type":"ContainerStarted","Data":"6e96a1e14e715ac5e6a23f9e16380de94647b471519414d13d95d8ac07cce982"}
Jul 06 18:18:08.534916 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:08.534883    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ztxm5" event={"ID":"0474b033-5051-4df3-8135-4b05741835ac","Type":"ContainerStarted","Data":"9f913882c18fcbdcbd989bc3ca9077de8ae8d783ba2ee34b0cd553b2aece0449"}
Jul 06 18:18:08.536469 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:08.536441    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-vz7bl" event={"ID":"af914b85-02d3-423a-aed5-e96b26e6c1c1","Type":"ContainerStarted","Data":"e95aea6b72b2028c0682631567b56d676eb87e91e73f8304ea1a3e09ce776802"}
Jul 06 18:18:08.548848 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:08.548797    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-113.ec2.internal" podStartSLOduration=22.548783231 podStartE2EDuration="22.548783231s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 18:17:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:18:08.548245645 +0000 UTC m=+23.705644132" watchObservedRunningTime="2026-07-06 18:18:08.548783231 +0000 UTC m=+23.706181716"
Jul 06 18:18:08.562960 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:08.562918    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-vz7bl" podStartSLOduration=3.81116842 podStartE2EDuration="23.562903416s" totalImagesPullingTime="19.751734996s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 18:17:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:18:08.562799325 +0000 UTC m=+23.720197810" watchObservedRunningTime="2026-07-06 18:18:08.562903416 +0000 UTC m=+23.720301902"
Jul 06 18:18:09.318087 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:09.318061    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xjbzr"
Jul 06 18:18:09.318248 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:09.318111    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fp8fn"
Jul 06 18:18:09.318248 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:18:09.318209    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xjbzr" podUID="f22387dc-2c30-4f80-8a80-3f6420c1c858"
Jul 06 18:18:09.318383 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:18:09.318331    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fp8fn" podUID="4e2f69c5-ef58-430a-8b74-eb53ce230c7e"
Jul 06 18:18:09.541968 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:09.541890    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9h479" event={"ID":"5cfed2e4-8993-42f8-bc84-f9399b287729","Type":"ContainerStarted","Data":"b38245e97ee3e7cc9b7c98631206c35c5b6defb135b09ff02e9a3111dd75971f"}
Jul 06 18:18:09.544179 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:09.544151    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ztxm5" event={"ID":"0474b033-5051-4df3-8135-4b05741835ac","Type":"ContainerStarted","Data":"f89a7d3c0d21ad1ada6ef854d48eb89bd481393ad6af91af91097ac9d1c970a3"}
Jul 06 18:18:09.564863 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:09.564806    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ztxm5" podStartSLOduration=2.7478571990000003 podStartE2EDuration="24.564786434s" totalImagesPullingTime="21.816929235s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 18:17:45 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:18:09.56421669 +0000 UTC m=+24.721615177" watchObservedRunningTime="2026-07-06 18:18:09.564786434 +0000 UTC m=+24.722184922"
Jul 06 18:18:10.316910 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:10.316874    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xzqch"
Jul 06 18:18:10.317182 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:18:10.316980    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xzqch" podUID="3b6b323e-eca7-4c68-87a7-0d48a9114a2c"
Jul 06 18:18:10.880184 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:10.880147    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-b7jm9"
Jul 06 18:18:10.880821 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:10.880739    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-b7jm9"
Jul 06 18:18:11.317076 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:11.316982    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fp8fn"
Jul 06 18:18:11.317076 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:11.316982    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xjbzr"
Jul 06 18:18:11.317298 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:18:11.317093    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fp8fn" podUID="4e2f69c5-ef58-430a-8b74-eb53ce230c7e"
Jul 06 18:18:11.317298 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:18:11.317239    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xjbzr" podUID="f22387dc-2c30-4f80-8a80-3f6420c1c858"
Jul 06 18:18:11.553152 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:11.552701    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9h479" event={"ID":"5cfed2e4-8993-42f8-bc84-f9399b287729","Type":"ContainerStarted","Data":"e87ebcb40d33055c5b9a679097a37186c61d93955ce6034b95828abffd8cf466"}
Jul 06 18:18:11.553152 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:11.553003    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-b7jm9"
Jul 06 18:18:11.553609 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:11.553545    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-b7jm9"
Jul 06 18:18:11.611868 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:11.611818    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-9h479" podStartSLOduration=6.76835792 podStartE2EDuration="26.611801994s" totalImagesPullingTime="19.843444074s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 18:17:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:18:11.596213638 +0000 UTC m=+26.753612124" watchObservedRunningTime="2026-07-06 18:18:11.611801994 +0000 UTC m=+26.769200480"
Jul 06 18:18:12.316909 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:12.316878    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xzqch"
Jul 06 18:18:12.317588 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:18:12.316994    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xzqch" podUID="3b6b323e-eca7-4c68-87a7-0d48a9114a2c"
Jul 06 18:18:12.555787 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:12.555757    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-9h479"
Jul 06 18:18:12.555968 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:12.555854    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-9h479"
Jul 06 18:18:12.555968 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:12.555903    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-9h479"
Jul 06 18:18:12.572381 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:12.572307    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-9h479"
Jul 06 18:18:12.572844 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:12.572770    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-9h479"
Jul 06 18:18:13.317671 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:13.317634    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xjbzr"
Jul 06 18:18:13.317671 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:13.317659    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fp8fn"
Jul 06 18:18:13.318349 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:18:13.317790    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xjbzr" podUID="f22387dc-2c30-4f80-8a80-3f6420c1c858"
Jul 06 18:18:13.318349 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:18:13.317922    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fp8fn" podUID="4e2f69c5-ef58-430a-8b74-eb53ce230c7e"
Jul 06 18:18:13.511057 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:13.510957    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-xzqch"]
Jul 06 18:18:13.511230 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:13.511119    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xzqch"
Jul 06 18:18:13.511290 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:18:13.511227    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xzqch" podUID="3b6b323e-eca7-4c68-87a7-0d48a9114a2c"
Jul 06 18:18:13.514874 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:13.514787    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-xjbzr"]
Jul 06 18:18:13.515271 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:13.515207    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-fp8fn"]
Jul 06 18:18:13.557658 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:13.557561    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fp8fn"
Jul 06 18:18:13.557783 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:18:13.557673    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fp8fn" podUID="4e2f69c5-ef58-430a-8b74-eb53ce230c7e"
Jul 06 18:18:13.558385 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:13.558356    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xjbzr"
Jul 06 18:18:13.558503 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:18:13.558477    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xjbzr" podUID="f22387dc-2c30-4f80-8a80-3f6420c1c858"
Jul 06 18:18:14.562102 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:14.561993    2580 generic.go:356] "Generic (PLEG): container finished" podID="a420108c-26d7-46d2-a3d5-3b354aa01af0" containerID="1a70f89daf36ef71daa31a823f0ea5e50bdadee0a878c92d0955455e4153e90a" exitCode=0
Jul 06 18:18:14.562102 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:14.562076    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-sl8sj" event={"ID":"a420108c-26d7-46d2-a3d5-3b354aa01af0","Type":"ContainerDied","Data":"1a70f89daf36ef71daa31a823f0ea5e50bdadee0a878c92d0955455e4153e90a"}
Jul 06 18:18:15.318553 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:15.318473    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fp8fn"
Jul 06 18:18:15.318553 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:15.318553    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xjbzr"
Jul 06 18:18:15.318706 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:15.318597    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xzqch"
Jul 06 18:18:15.318706 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:18:15.318618    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fp8fn" podUID="4e2f69c5-ef58-430a-8b74-eb53ce230c7e"
Jul 06 18:18:15.318706 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:18:15.318683    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xzqch" podUID="3b6b323e-eca7-4c68-87a7-0d48a9114a2c"
Jul 06 18:18:15.318849 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:18:15.318785    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xjbzr" podUID="f22387dc-2c30-4f80-8a80-3f6420c1c858"
Jul 06 18:18:15.567101 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:15.567028    2580 generic.go:356] "Generic (PLEG): container finished" podID="a420108c-26d7-46d2-a3d5-3b354aa01af0" containerID="d90206e96dbf96016f39f4d6a0341fb650031b29fadd1197c8814a14038a7686" exitCode=0
Jul 06 18:18:15.567101 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:15.567089    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-sl8sj" event={"ID":"a420108c-26d7-46d2-a3d5-3b354aa01af0","Type":"ContainerDied","Data":"d90206e96dbf96016f39f4d6a0341fb650031b29fadd1197c8814a14038a7686"}
Jul 06 18:18:16.571603 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:16.571573    2580 generic.go:356] "Generic (PLEG): container finished" podID="a420108c-26d7-46d2-a3d5-3b354aa01af0" containerID="f73fe2473a3cea1150c34d6e7362e24874b6885b80c526bbc13d8024870eaae5" exitCode=0
Jul 06 18:18:16.571930 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:16.571617    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-sl8sj" event={"ID":"a420108c-26d7-46d2-a3d5-3b354aa01af0","Type":"ContainerDied","Data":"f73fe2473a3cea1150c34d6e7362e24874b6885b80c526bbc13d8024870eaae5"}
Jul 06 18:18:17.317777 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:17.317752    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fp8fn"
Jul 06 18:18:17.317777 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:17.317767    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xjbzr"
Jul 06 18:18:17.317962 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:18:17.317846    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fp8fn" podUID="4e2f69c5-ef58-430a-8b74-eb53ce230c7e"
Jul 06 18:18:17.318547 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:18:17.318334    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xjbzr" podUID="f22387dc-2c30-4f80-8a80-3f6420c1c858"
Jul 06 18:18:17.318547 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:17.318482    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xzqch"
Jul 06 18:18:17.321035 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:18:17.318912    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xzqch" podUID="3b6b323e-eca7-4c68-87a7-0d48a9114a2c"
Jul 06 18:18:17.950501 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:17.950466    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-knzwx\" (UniqueName: \"kubernetes.io/projected/4e2f69c5-ef58-430a-8b74-eb53ce230c7e-kube-api-access-knzwx\") pod \"network-check-target-fp8fn\" (UID: \"4e2f69c5-ef58-430a-8b74-eb53ce230c7e\") " pod="openshift-network-diagnostics/network-check-target-fp8fn"
Jul 06 18:18:17.950952 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:18:17.950654    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 06 18:18:17.950952 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:18:17.950680    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 06 18:18:17.950952 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:18:17.950692    2580 projected.go:196] Error preparing data for projected volume kube-api-access-knzwx for pod openshift-network-diagnostics/network-check-target-fp8fn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 18:18:17.950952 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:18:17.950755    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4e2f69c5-ef58-430a-8b74-eb53ce230c7e-kube-api-access-knzwx podName:4e2f69c5-ef58-430a-8b74-eb53ce230c7e nodeName:}" failed. No retries permitted until 2026-07-06 18:18:49.950735899 +0000 UTC m=+65.108134362 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-knzwx" (UniqueName: "kubernetes.io/projected/4e2f69c5-ef58-430a-8b74-eb53ce230c7e-kube-api-access-knzwx") pod "network-check-target-fp8fn" (UID: "4e2f69c5-ef58-430a-8b74-eb53ce230c7e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 18:18:18.051135 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:18.051103    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3b6b323e-eca7-4c68-87a7-0d48a9114a2c-original-pull-secret\") pod \"global-pull-secret-syncer-xzqch\" (UID: \"3b6b323e-eca7-4c68-87a7-0d48a9114a2c\") " pod="kube-system/global-pull-secret-syncer-xzqch"
Jul 06 18:18:18.051135 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:18.051141    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f22387dc-2c30-4f80-8a80-3f6420c1c858-metrics-certs\") pod \"network-metrics-daemon-xjbzr\" (UID: \"f22387dc-2c30-4f80-8a80-3f6420c1c858\") " pod="openshift-multus/network-metrics-daemon-xjbzr"
Jul 06 18:18:18.051358 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:18:18.051263    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 06 18:18:18.051358 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:18:18.051282    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 18:18:18.051358 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:18:18.051330    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b6b323e-eca7-4c68-87a7-0d48a9114a2c-original-pull-secret podName:3b6b323e-eca7-4c68-87a7-0d48a9114a2c nodeName:}" failed. No retries permitted until 2026-07-06 18:18:50.051313576 +0000 UTC m=+65.208712039 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3b6b323e-eca7-4c68-87a7-0d48a9114a2c-original-pull-secret") pod "global-pull-secret-syncer-xzqch" (UID: "3b6b323e-eca7-4c68-87a7-0d48a9114a2c") : object "kube-system"/"original-pull-secret" not registered
Jul 06 18:18:18.051358 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:18:18.051345    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f22387dc-2c30-4f80-8a80-3f6420c1c858-metrics-certs podName:f22387dc-2c30-4f80-8a80-3f6420c1c858 nodeName:}" failed. No retries permitted until 2026-07-06 18:18:50.051337896 +0000 UTC m=+65.208736360 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f22387dc-2c30-4f80-8a80-3f6420c1c858-metrics-certs") pod "network-metrics-daemon-xjbzr" (UID: "f22387dc-2c30-4f80-8a80-3f6420c1c858") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 18:18:19.317798 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:19.317768    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xzqch"
Jul 06 18:18:19.318244 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:19.317768    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fp8fn"
Jul 06 18:18:19.318244 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:18:19.317887    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xzqch" podUID="3b6b323e-eca7-4c68-87a7-0d48a9114a2c"
Jul 06 18:18:19.318244 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:19.317768    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xjbzr"
Jul 06 18:18:19.318244 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:18:19.317971    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fp8fn" podUID="4e2f69c5-ef58-430a-8b74-eb53ce230c7e"
Jul 06 18:18:19.318244 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:18:19.318067    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xjbzr" podUID="f22387dc-2c30-4f80-8a80-3f6420c1c858"
Jul 06 18:18:20.158692 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:20.158665    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-113.ec2.internal" event="NodeReady"
Jul 06 18:18:20.158854 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:20.158786    2580 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 06 18:18:20.194136 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:20.194059    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-58c4895898-bt6wr"]
Jul 06 18:18:20.208778 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:20.208750    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-nmspz"]
Jul 06 18:18:20.208911 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:20.208880    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-58c4895898-bt6wr"
Jul 06 18:18:20.211986 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:20.211943    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 06 18:18:20.212158 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:20.212045    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 06 18:18:20.212665 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:20.212640    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 06 18:18:20.213545 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:20.213049    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-j279m\""
Jul 06 18:18:20.220839 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:20.220622    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 06 18:18:20.224874 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:20.224850    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-89h8f"]
Jul 06 18:18:20.224984 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:20.224952    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-nmspz"
Jul 06 18:18:20.228832 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:20.228809    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 06 18:18:20.229067 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:20.228969    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 06 18:18:20.229202 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:20.228904    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-w2z5d\""
Jul 06 18:18:20.229202 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:20.229098    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 06 18:18:20.242898 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:20.242870    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-58c4895898-bt6wr"]
Jul 06 18:18:20.243008 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:20.242912    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-nmspz"]
Jul 06 18:18:20.243008 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:20.242936    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-89h8f"]
Jul 06 18:18:20.243008 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:20.242969    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-lzzjn"]
Jul 06 18:18:20.243008 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:20.242974    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-89h8f"
Jul 06 18:18:20.246627 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:20.246603    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-6mvj2\""
Jul 06 18:18:20.246757 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:20.246647    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 06 18:18:20.247263 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:20.247239    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 06 18:18:20.247554 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:20.247536    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 06 18:18:20.247554 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:20.247552    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 06 18:18:20.269114 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:20.269088    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-lzzjn"]
Jul 06 18:18:20.269241 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:20.269221    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-lzzjn"
Jul 06 18:18:20.271829 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:20.271810    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 06 18:18:20.271829 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:20.271820    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-b25k8\""
Jul 06 18:18:20.272301 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:20.272287    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 06 18:18:20.366979 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:20.366941    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f354e957-d479-41ba-bf43-24303d500049-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-89h8f\" (UID: \"f354e957-d479-41ba-bf43-24303d500049\") " pod="openshift-insights/insights-runtime-extractor-89h8f"
Jul 06 18:18:20.367419 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:20.367005    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/24fc32df-2fce-41bf-8a8d-849b392bcb8f-tmp-dir\") pod \"dns-default-lzzjn\" (UID: \"24fc32df-2fce-41bf-8a8d-849b392bcb8f\") " pod="openshift-dns/dns-default-lzzjn"
Jul 06 18:18:20.367419 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:20.367054    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/cc40fe47-16bf-4a41-90e6-29f7e89f687f-ca-trust-extracted\") pod \"image-registry-58c4895898-bt6wr\" (UID: \"cc40fe47-16bf-4a41-90e6-29f7e89f687f\") " pod="openshift-image-registry/image-registry-58c4895898-bt6wr"
Jul 06 18:18:20.367419 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:20.367072    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cc40fe47-16bf-4a41-90e6-29f7e89f687f-bound-sa-token\") pod \"image-registry-58c4895898-bt6wr\" (UID: \"cc40fe47-16bf-4a41-90e6-29f7e89f687f\") " pod="openshift-image-registry/image-registry-58c4895898-bt6wr"
Jul 06 18:18:20.367419 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:20.367089    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f354e957-d479-41ba-bf43-24303d500049-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-89h8f\" (UID: \"f354e957-d479-41ba-bf43-24303d500049\") " pod="openshift-insights/insights-runtime-extractor-89h8f"
Jul 06 18:18:20.367419 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:20.367107    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-md57k\" (UniqueName: \"kubernetes.io/projected/534f2c50-cd67-4d03-90b9-5076223138ff-kube-api-access-md57k\") pod \"ingress-canary-nmspz\" (UID: \"534f2c50-cd67-4d03-90b9-5076223138ff\") " pod="openshift-ingress-canary/ingress-canary-nmspz"
Jul 06 18:18:20.367419 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:20.367124    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/24fc32df-2fce-41bf-8a8d-849b392bcb8f-config-volume\") pod \"dns-default-lzzjn\" (UID: \"24fc32df-2fce-41bf-8a8d-849b392bcb8f\") " pod="openshift-dns/dns-default-lzzjn"
Jul 06 18:18:20.367419 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:20.367214    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/534f2c50-cd67-4d03-90b9-5076223138ff-cert\") pod \"ingress-canary-nmspz\" (UID: \"534f2c50-cd67-4d03-90b9-5076223138ff\") " pod="openshift-ingress-canary/ingress-canary-nmspz"
Jul 06 18:18:20.367419 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:20.367240    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/cc40fe47-16bf-4a41-90e6-29f7e89f687f-image-registry-private-configuration\") pod \"image-registry-58c4895898-bt6wr\" (UID: \"cc40fe47-16bf-4a41-90e6-29f7e89f687f\") " pod="openshift-image-registry/image-registry-58c4895898-bt6wr"
Jul 06 18:18:20.367419 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:20.367261    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/cc40fe47-16bf-4a41-90e6-29f7e89f687f-installation-pull-secrets\") pod \"image-registry-58c4895898-bt6wr\" (UID: \"cc40fe47-16bf-4a41-90e6-29f7e89f687f\") " pod="openshift-image-registry/image-registry-58c4895898-bt6wr"
Jul 06 18:18:20.367419 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:20.367277    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f354e957-d479-41ba-bf43-24303d500049-data-volume\") pod \"insights-runtime-extractor-89h8f\" (UID: \"f354e957-d479-41ba-bf43-24303d500049\") " pod="openshift-insights/insights-runtime-extractor-89h8f"
Jul 06 18:18:20.367419 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:20.367315    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cc40fe47-16bf-4a41-90e6-29f7e89f687f-registry-tls\") pod \"image-registry-58c4895898-bt6wr\" (UID: \"cc40fe47-16bf-4a41-90e6-29f7e89f687f\") " pod="openshift-image-registry/image-registry-58c4895898-bt6wr"
Jul 06 18:18:20.367419 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:20.367330    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sldnn\" (UniqueName: \"kubernetes.io/projected/f354e957-d479-41ba-bf43-24303d500049-kube-api-access-sldnn\") pod \"insights-runtime-extractor-89h8f\" (UID: \"f354e957-d479-41ba-bf43-24303d500049\") " pod="openshift-insights/insights-runtime-extractor-89h8f"
Jul 06 18:18:20.367419 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:20.367349    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/cc40fe47-16bf-4a41-90e6-29f7e89f687f-registry-certificates\") pod \"image-registry-58c4895898-bt6wr\" (UID: \"cc40fe47-16bf-4a41-90e6-29f7e89f687f\") " pod="openshift-image-registry/image-registry-58c4895898-bt6wr"
Jul 06 18:18:20.367419 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:20.367369    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4slp2\" (UniqueName: \"kubernetes.io/projected/24fc32df-2fce-41bf-8a8d-849b392bcb8f-kube-api-access-4slp2\") pod \"dns-default-lzzjn\" (UID: \"24fc32df-2fce-41bf-8a8d-849b392bcb8f\") " pod="openshift-dns/dns-default-lzzjn"
Jul 06 18:18:20.367419 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:20.367383    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cc40fe47-16bf-4a41-90e6-29f7e89f687f-trusted-ca\") pod \"image-registry-58c4895898-bt6wr\" (UID: \"cc40fe47-16bf-4a41-90e6-29f7e89f687f\") " pod="openshift-image-registry/image-registry-58c4895898-bt6wr"
Jul 06 18:18:20.367867 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:20.367399    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n78h6\" (UniqueName: \"kubernetes.io/projected/cc40fe47-16bf-4a41-90e6-29f7e89f687f-kube-api-access-n78h6\") pod \"image-registry-58c4895898-bt6wr\" (UID: \"cc40fe47-16bf-4a41-90e6-29f7e89f687f\") " pod="openshift-image-registry/image-registry-58c4895898-bt6wr"
Jul 06 18:18:20.367867 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:20.367414    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f354e957-d479-41ba-bf43-24303d500049-crio-socket\") pod \"insights-runtime-extractor-89h8f\" (UID: \"f354e957-d479-41ba-bf43-24303d500049\") " pod="openshift-insights/insights-runtime-extractor-89h8f"
Jul 06 18:18:20.367867 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:20.367435    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/24fc32df-2fce-41bf-8a8d-849b392bcb8f-metrics-tls\") pod \"dns-default-lzzjn\" (UID: \"24fc32df-2fce-41bf-8a8d-849b392bcb8f\") " pod="openshift-dns/dns-default-lzzjn"
Jul 06 18:18:20.468302 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:20.468214    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/534f2c50-cd67-4d03-90b9-5076223138ff-cert\") pod \"ingress-canary-nmspz\" (UID: \"534f2c50-cd67-4d03-90b9-5076223138ff\") " pod="openshift-ingress-canary/ingress-canary-nmspz"
Jul 06 18:18:20.468302 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:20.468253    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/cc40fe47-16bf-4a41-90e6-29f7e89f687f-image-registry-private-configuration\") pod \"image-registry-58c4895898-bt6wr\" (UID: \"cc40fe47-16bf-4a41-90e6-29f7e89f687f\") " pod="openshift-image-registry/image-registry-58c4895898-bt6wr"
Jul 06 18:18:20.468302 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:20.468284    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/cc40fe47-16bf-4a41-90e6-29f7e89f687f-installation-pull-secrets\") pod \"image-registry-58c4895898-bt6wr\" (UID: \"cc40fe47-16bf-4a41-90e6-29f7e89f687f\") " pod="openshift-image-registry/image-registry-58c4895898-bt6wr"
Jul 06 18:18:20.468552 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:20.468423    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f354e957-d479-41ba-bf43-24303d500049-data-volume\") pod \"insights-runtime-extractor-89h8f\" (UID: \"f354e957-d479-41ba-bf43-24303d500049\") " pod="openshift-insights/insights-runtime-extractor-89h8f"
Jul 06 18:18:20.468552 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:20.468472    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cc40fe47-16bf-4a41-90e6-29f7e89f687f-registry-tls\") pod \"image-registry-58c4895898-bt6wr\" (UID: \"cc40fe47-16bf-4a41-90e6-29f7e89f687f\") " pod="openshift-image-registry/image-registry-58c4895898-bt6wr"
Jul 06 18:18:20.468552 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:20.468490    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sldnn\" (UniqueName: \"kubernetes.io/projected/f354e957-d479-41ba-bf43-24303d500049-kube-api-access-sldnn\") pod \"insights-runtime-extractor-89h8f\" (UID: \"f354e957-d479-41ba-bf43-24303d500049\") " pod="openshift-insights/insights-runtime-extractor-89h8f"
Jul 06 18:18:20.468552 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:20.468513    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/cc40fe47-16bf-4a41-90e6-29f7e89f687f-registry-certificates\") pod \"image-registry-58c4895898-bt6wr\" (UID: \"cc40fe47-16bf-4a41-90e6-29f7e89f687f\") " pod="openshift-image-registry/image-registry-58c4895898-bt6wr"
Jul 06 18:18:20.468756 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:18:20.468589    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 06 18:18:20.468756 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:18:20.468606    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-58c4895898-bt6wr: secret "image-registry-tls" not found
Jul 06 18:18:20.468756 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:18:20.468670    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cc40fe47-16bf-4a41-90e6-29f7e89f687f-registry-tls podName:cc40fe47-16bf-4a41-90e6-29f7e89f687f nodeName:}" failed. No retries permitted until 2026-07-06 18:18:20.968652799 +0000 UTC m=+36.126051282 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/cc40fe47-16bf-4a41-90e6-29f7e89f687f-registry-tls") pod "image-registry-58c4895898-bt6wr" (UID: "cc40fe47-16bf-4a41-90e6-29f7e89f687f") : secret "image-registry-tls" not found
Jul 06 18:18:20.468944 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:20.468921    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4slp2\" (UniqueName: \"kubernetes.io/projected/24fc32df-2fce-41bf-8a8d-849b392bcb8f-kube-api-access-4slp2\") pod \"dns-default-lzzjn\" (UID: \"24fc32df-2fce-41bf-8a8d-849b392bcb8f\") " pod="openshift-dns/dns-default-lzzjn"
Jul 06 18:18:20.468986 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:20.468967    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cc40fe47-16bf-4a41-90e6-29f7e89f687f-trusted-ca\") pod \"image-registry-58c4895898-bt6wr\" (UID: \"cc40fe47-16bf-4a41-90e6-29f7e89f687f\") " pod="openshift-image-registry/image-registry-58c4895898-bt6wr"
Jul 06 18:18:20.469040 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:20.469002    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n78h6\" (UniqueName: \"kubernetes.io/projected/cc40fe47-16bf-4a41-90e6-29f7e89f687f-kube-api-access-n78h6\") pod \"image-registry-58c4895898-bt6wr\" (UID: \"cc40fe47-16bf-4a41-90e6-29f7e89f687f\") " pod="openshift-image-registry/image-registry-58c4895898-bt6wr"
Jul 06 18:18:20.469081 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:20.469065    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f354e957-d479-41ba-bf43-24303d500049-crio-socket\") pod \"insights-runtime-extractor-89h8f\" (UID: \"f354e957-d479-41ba-bf43-24303d500049\") " pod="openshift-insights/insights-runtime-extractor-89h8f"
Jul 06 18:18:20.469113 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:20.469094    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/24fc32df-2fce-41bf-8a8d-849b392bcb8f-metrics-tls\") pod \"dns-default-lzzjn\" (UID: \"24fc32df-2fce-41bf-8a8d-849b392bcb8f\") " pod="openshift-dns/dns-default-lzzjn"
Jul 06 18:18:20.469158 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:20.469138    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f354e957-d479-41ba-bf43-24303d500049-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-89h8f\" (UID: \"f354e957-d479-41ba-bf43-24303d500049\") " pod="openshift-insights/insights-runtime-extractor-89h8f"
Jul 06 18:18:20.469242 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:20.469226    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/24fc32df-2fce-41bf-8a8d-849b392bcb8f-tmp-dir\") pod \"dns-default-lzzjn\" (UID: \"24fc32df-2fce-41bf-8a8d-849b392bcb8f\") " pod="openshift-dns/dns-default-lzzjn"
Jul 06 18:18:20.469295 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:20.469257    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/cc40fe47-16bf-4a41-90e6-29f7e89f687f-registry-certificates\") pod \"image-registry-58c4895898-bt6wr\" (UID: \"cc40fe47-16bf-4a41-90e6-29f7e89f687f\") " pod="openshift-image-registry/image-registry-58c4895898-bt6wr"
Jul 06 18:18:20.469295 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:20.469263    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/cc40fe47-16bf-4a41-90e6-29f7e89f687f-ca-trust-extracted\") pod \"image-registry-58c4895898-bt6wr\" (UID: \"cc40fe47-16bf-4a41-90e6-29f7e89f687f\") " pod="openshift-image-registry/image-registry-58c4895898-bt6wr"
Jul 06 18:18:20.469295 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:20.469287    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cc40fe47-16bf-4a41-90e6-29f7e89f687f-bound-sa-token\") pod \"image-registry-58c4895898-bt6wr\" (UID: \"cc40fe47-16bf-4a41-90e6-29f7e89f687f\") " pod="openshift-image-registry/image-registry-58c4895898-bt6wr"
Jul 06 18:18:20.469428 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:20.469316    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f354e957-d479-41ba-bf43-24303d500049-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-89h8f\" (UID: \"f354e957-d479-41ba-bf43-24303d500049\") " pod="openshift-insights/insights-runtime-extractor-89h8f"
Jul 06 18:18:20.469428 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:20.469347    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-md57k\" (UniqueName: \"kubernetes.io/projected/534f2c50-cd67-4d03-90b9-5076223138ff-kube-api-access-md57k\") pod \"ingress-canary-nmspz\" (UID: \"534f2c50-cd67-4d03-90b9-5076223138ff\") " pod="openshift-ingress-canary/ingress-canary-nmspz"
Jul 06 18:18:20.469428 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:18:20.469352    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 06 18:18:20.469428 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:18:20.469364    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 06 18:18:20.469428 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:20.469369    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/24fc32df-2fce-41bf-8a8d-849b392bcb8f-config-volume\") pod \"dns-default-lzzjn\" (UID: \"24fc32df-2fce-41bf-8a8d-849b392bcb8f\") " pod="openshift-dns/dns-default-lzzjn"
Jul 06 18:18:20.469428 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:18:20.469400    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f354e957-d479-41ba-bf43-24303d500049-insights-runtime-extractor-tls podName:f354e957-d479-41ba-bf43-24303d500049 nodeName:}" failed. No retries permitted until 2026-07-06 18:18:20.969385792 +0000 UTC m=+36.126784278 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f354e957-d479-41ba-bf43-24303d500049-insights-runtime-extractor-tls") pod "insights-runtime-extractor-89h8f" (UID: "f354e957-d479-41ba-bf43-24303d500049") : secret "insights-runtime-extractor-tls" not found
Jul 06 18:18:20.469428 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:20.469262    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f354e957-d479-41ba-bf43-24303d500049-data-volume\") pod \"insights-runtime-extractor-89h8f\" (UID: \"f354e957-d479-41ba-bf43-24303d500049\") " pod="openshift-insights/insights-runtime-extractor-89h8f"
Jul 06 18:18:20.469428 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:18:20.469416    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24fc32df-2fce-41bf-8a8d-849b392bcb8f-metrics-tls podName:24fc32df-2fce-41bf-8a8d-849b392bcb8f nodeName:}" failed. No retries permitted until 2026-07-06 18:18:20.969408749 +0000 UTC m=+36.126807213 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/24fc32df-2fce-41bf-8a8d-849b392bcb8f-metrics-tls") pod "dns-default-lzzjn" (UID: "24fc32df-2fce-41bf-8a8d-849b392bcb8f") : secret "dns-default-metrics-tls" not found
Jul 06 18:18:20.469817 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:18:20.469476    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 06 18:18:20.469817 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:18:20.469522    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f354e957-d479-41ba-bf43-24303d500049-kube-rbac-proxy-cm podName:f354e957-d479-41ba-bf43-24303d500049 nodeName:}" failed. No retries permitted until 2026-07-06 18:18:20.969508347 +0000 UTC m=+36.126906833 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f354e957-d479-41ba-bf43-24303d500049-kube-rbac-proxy-cm") pod "insights-runtime-extractor-89h8f" (UID: "f354e957-d479-41ba-bf43-24303d500049") : configmap "kube-rbac-proxy" not found
Jul 06 18:18:20.469817 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:20.469573    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f354e957-d479-41ba-bf43-24303d500049-crio-socket\") pod \"insights-runtime-extractor-89h8f\" (UID: \"f354e957-d479-41ba-bf43-24303d500049\") " pod="openshift-insights/insights-runtime-extractor-89h8f"
Jul 06 18:18:20.469817 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:20.469666    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/24fc32df-2fce-41bf-8a8d-849b392bcb8f-tmp-dir\") pod \"dns-default-lzzjn\" (UID: \"24fc32df-2fce-41bf-8a8d-849b392bcb8f\") " pod="openshift-dns/dns-default-lzzjn"
Jul 06 18:18:20.469817 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:20.469712    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/cc40fe47-16bf-4a41-90e6-29f7e89f687f-ca-trust-extracted\") pod \"image-registry-58c4895898-bt6wr\" (UID: \"cc40fe47-16bf-4a41-90e6-29f7e89f687f\") " pod="openshift-image-registry/image-registry-58c4895898-bt6wr"
Jul 06 18:18:20.470148 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:20.470123    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/24fc32df-2fce-41bf-8a8d-849b392bcb8f-config-volume\") pod \"dns-default-lzzjn\" (UID: \"24fc32df-2fce-41bf-8a8d-849b392bcb8f\") " pod="openshift-dns/dns-default-lzzjn"
Jul 06 18:18:20.470504 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:20.470471    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cc40fe47-16bf-4a41-90e6-29f7e89f687f-trusted-ca\") pod \"image-registry-58c4895898-bt6wr\" (UID: \"cc40fe47-16bf-4a41-90e6-29f7e89f687f\") " pod="openshift-image-registry/image-registry-58c4895898-bt6wr"
Jul 06 18:18:20.472884 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:20.472861    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/534f2c50-cd67-4d03-90b9-5076223138ff-cert\") pod \"ingress-canary-nmspz\" (UID: \"534f2c50-cd67-4d03-90b9-5076223138ff\") " pod="openshift-ingress-canary/ingress-canary-nmspz"
Jul 06 18:18:20.472972 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:20.472884    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/cc40fe47-16bf-4a41-90e6-29f7e89f687f-image-registry-private-configuration\") pod \"image-registry-58c4895898-bt6wr\" (UID: \"cc40fe47-16bf-4a41-90e6-29f7e89f687f\") " pod="openshift-image-registry/image-registry-58c4895898-bt6wr"
Jul 06 18:18:20.472972 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:20.472884    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/cc40fe47-16bf-4a41-90e6-29f7e89f687f-installation-pull-secrets\") pod \"image-registry-58c4895898-bt6wr\" (UID: \"cc40fe47-16bf-4a41-90e6-29f7e89f687f\") " pod="openshift-image-registry/image-registry-58c4895898-bt6wr"
Jul 06 18:18:20.479891 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:20.479871    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cc40fe47-16bf-4a41-90e6-29f7e89f687f-bound-sa-token\") pod \"image-registry-58c4895898-bt6wr\" (UID: \"cc40fe47-16bf-4a41-90e6-29f7e89f687f\") " pod="openshift-image-registry/image-registry-58c4895898-bt6wr"
Jul 06 18:18:20.480280 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:20.480259    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4slp2\" (UniqueName: \"kubernetes.io/projected/24fc32df-2fce-41bf-8a8d-849b392bcb8f-kube-api-access-4slp2\") pod \"dns-default-lzzjn\" (UID: \"24fc32df-2fce-41bf-8a8d-849b392bcb8f\") " pod="openshift-dns/dns-default-lzzjn"
Jul 06 18:18:20.480425 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:20.480406    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sldnn\" (UniqueName: \"kubernetes.io/projected/f354e957-d479-41ba-bf43-24303d500049-kube-api-access-sldnn\") pod \"insights-runtime-extractor-89h8f\" (UID: \"f354e957-d479-41ba-bf43-24303d500049\") " pod="openshift-insights/insights-runtime-extractor-89h8f"
Jul 06 18:18:20.481058 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:20.480962    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-md57k\" (UniqueName: \"kubernetes.io/projected/534f2c50-cd67-4d03-90b9-5076223138ff-kube-api-access-md57k\") pod \"ingress-canary-nmspz\" (UID: \"534f2c50-cd67-4d03-90b9-5076223138ff\") " pod="openshift-ingress-canary/ingress-canary-nmspz"
Jul 06 18:18:20.481429 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:20.481406    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n78h6\" (UniqueName: \"kubernetes.io/projected/cc40fe47-16bf-4a41-90e6-29f7e89f687f-kube-api-access-n78h6\") pod \"image-registry-58c4895898-bt6wr\" (UID: \"cc40fe47-16bf-4a41-90e6-29f7e89f687f\") " pod="openshift-image-registry/image-registry-58c4895898-bt6wr"
Jul 06 18:18:20.535491 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:20.535460    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-nmspz"
Jul 06 18:18:20.735302 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:20.735234    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-nmspz"]
Jul 06 18:18:20.738799 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:18:20.738765    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod534f2c50_cd67_4d03_90b9_5076223138ff.slice/crio-19318b9f1802fb76661231bc4e3510c80ef86b6f6395186dea7446e0613f8db8 WatchSource:0}: Error finding container 19318b9f1802fb76661231bc4e3510c80ef86b6f6395186dea7446e0613f8db8: Status 404 returned error can't find the container with id 19318b9f1802fb76661231bc4e3510c80ef86b6f6395186dea7446e0613f8db8
Jul 06 18:18:20.974522 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:20.974487    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/24fc32df-2fce-41bf-8a8d-849b392bcb8f-metrics-tls\") pod \"dns-default-lzzjn\" (UID: \"24fc32df-2fce-41bf-8a8d-849b392bcb8f\") " pod="openshift-dns/dns-default-lzzjn"
Jul 06 18:18:20.974709 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:20.974541    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f354e957-d479-41ba-bf43-24303d500049-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-89h8f\" (UID: \"f354e957-d479-41ba-bf43-24303d500049\") " pod="openshift-insights/insights-runtime-extractor-89h8f"
Jul 06 18:18:20.974709 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:20.974614    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f354e957-d479-41ba-bf43-24303d500049-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-89h8f\" (UID: \"f354e957-d479-41ba-bf43-24303d500049\") " pod="openshift-insights/insights-runtime-extractor-89h8f"
Jul 06 18:18:20.974709 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:18:20.974650    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 06 18:18:20.974709 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:20.974675    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cc40fe47-16bf-4a41-90e6-29f7e89f687f-registry-tls\") pod \"image-registry-58c4895898-bt6wr\" (UID: \"cc40fe47-16bf-4a41-90e6-29f7e89f687f\") " pod="openshift-image-registry/image-registry-58c4895898-bt6wr"
Jul 06 18:18:20.974881 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:18:20.974715    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 06 18:18:20.974881 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:18:20.974721    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24fc32df-2fce-41bf-8a8d-849b392bcb8f-metrics-tls podName:24fc32df-2fce-41bf-8a8d-849b392bcb8f nodeName:}" failed. No retries permitted until 2026-07-06 18:18:21.974701275 +0000 UTC m=+37.132099744 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/24fc32df-2fce-41bf-8a8d-849b392bcb8f-metrics-tls") pod "dns-default-lzzjn" (UID: "24fc32df-2fce-41bf-8a8d-849b392bcb8f") : secret "dns-default-metrics-tls" not found
Jul 06 18:18:20.974881 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:18:20.974777    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 06 18:18:20.974881 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:18:20.974788    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-58c4895898-bt6wr: secret "image-registry-tls" not found
Jul 06 18:18:20.974881 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:18:20.974791    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f354e957-d479-41ba-bf43-24303d500049-insights-runtime-extractor-tls podName:f354e957-d479-41ba-bf43-24303d500049 nodeName:}" failed. No retries permitted until 2026-07-06 18:18:21.974773488 +0000 UTC m=+37.132171956 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f354e957-d479-41ba-bf43-24303d500049-insights-runtime-extractor-tls") pod "insights-runtime-extractor-89h8f" (UID: "f354e957-d479-41ba-bf43-24303d500049") : secret "insights-runtime-extractor-tls" not found
Jul 06 18:18:20.974881 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:18:20.974721    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 06 18:18:20.974881 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:18:20.974833    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f354e957-d479-41ba-bf43-24303d500049-kube-rbac-proxy-cm podName:f354e957-d479-41ba-bf43-24303d500049 nodeName:}" failed. No retries permitted until 2026-07-06 18:18:21.97481932 +0000 UTC m=+37.132217787 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f354e957-d479-41ba-bf43-24303d500049-kube-rbac-proxy-cm") pod "insights-runtime-extractor-89h8f" (UID: "f354e957-d479-41ba-bf43-24303d500049") : configmap "kube-rbac-proxy" not found
Jul 06 18:18:20.974881 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:18:20.974850    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cc40fe47-16bf-4a41-90e6-29f7e89f687f-registry-tls podName:cc40fe47-16bf-4a41-90e6-29f7e89f687f nodeName:}" failed. No retries permitted until 2026-07-06 18:18:21.974841532 +0000 UTC m=+37.132239999 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/cc40fe47-16bf-4a41-90e6-29f7e89f687f-registry-tls") pod "image-registry-58c4895898-bt6wr" (UID: "cc40fe47-16bf-4a41-90e6-29f7e89f687f") : secret "image-registry-tls" not found
Jul 06 18:18:21.118345 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:21.118315    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-x7rd5_d27a6ce1-0910-4c08-959a-550b8d468219/dns-node-resolver/0.log"
Jul 06 18:18:21.317379 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:21.317345    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fp8fn"
Jul 06 18:18:21.317535 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:21.317350    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xjbzr"
Jul 06 18:18:21.317697 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:21.317662    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xzqch"
Jul 06 18:18:21.321998 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:21.321210    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 06 18:18:21.322575 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:21.322551    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 06 18:18:21.322695 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:21.322551    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-4lbvz\""
Jul 06 18:18:21.322695 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:21.322555    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 06 18:18:21.322695 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:21.322555    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-m8lgm\""
Jul 06 18:18:21.322695 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:21.322559    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 06 18:18:21.584782 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:21.584751    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-nmspz" event={"ID":"534f2c50-cd67-4d03-90b9-5076223138ff","Type":"ContainerStarted","Data":"19318b9f1802fb76661231bc4e3510c80ef86b6f6395186dea7446e0613f8db8"}
Jul 06 18:18:21.983554 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:21.983468    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cc40fe47-16bf-4a41-90e6-29f7e89f687f-registry-tls\") pod \"image-registry-58c4895898-bt6wr\" (UID: \"cc40fe47-16bf-4a41-90e6-29f7e89f687f\") " pod="openshift-image-registry/image-registry-58c4895898-bt6wr"
Jul 06 18:18:21.983554 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:21.983509    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/24fc32df-2fce-41bf-8a8d-849b392bcb8f-metrics-tls\") pod \"dns-default-lzzjn\" (UID: \"24fc32df-2fce-41bf-8a8d-849b392bcb8f\") " pod="openshift-dns/dns-default-lzzjn"
Jul 06 18:18:21.983554 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:21.983534    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f354e957-d479-41ba-bf43-24303d500049-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-89h8f\" (UID: \"f354e957-d479-41ba-bf43-24303d500049\") " pod="openshift-insights/insights-runtime-extractor-89h8f"
Jul 06 18:18:21.983820 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:18:21.983619    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 06 18:18:21.983820 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:18:21.983630    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 06 18:18:21.983820 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:18:21.983666    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 06 18:18:21.983820 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:21.983638    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f354e957-d479-41ba-bf43-24303d500049-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-89h8f\" (UID: \"f354e957-d479-41ba-bf43-24303d500049\") " pod="openshift-insights/insights-runtime-extractor-89h8f"
Jul 06 18:18:21.983820 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:18:21.983639    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-58c4895898-bt6wr: secret "image-registry-tls" not found
Jul 06 18:18:21.983820 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:18:21.983704    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24fc32df-2fce-41bf-8a8d-849b392bcb8f-metrics-tls podName:24fc32df-2fce-41bf-8a8d-849b392bcb8f nodeName:}" failed. No retries permitted until 2026-07-06 18:18:23.983684204 +0000 UTC m=+39.141082668 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/24fc32df-2fce-41bf-8a8d-849b392bcb8f-metrics-tls") pod "dns-default-lzzjn" (UID: "24fc32df-2fce-41bf-8a8d-849b392bcb8f") : secret "dns-default-metrics-tls" not found
Jul 06 18:18:21.983820 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:18:21.983722    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f354e957-d479-41ba-bf43-24303d500049-kube-rbac-proxy-cm podName:f354e957-d479-41ba-bf43-24303d500049 nodeName:}" failed. No retries permitted until 2026-07-06 18:18:23.983714101 +0000 UTC m=+39.141112572 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f354e957-d479-41ba-bf43-24303d500049-kube-rbac-proxy-cm") pod "insights-runtime-extractor-89h8f" (UID: "f354e957-d479-41ba-bf43-24303d500049") : configmap "kube-rbac-proxy" not found
Jul 06 18:18:21.983820 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:18:21.983624    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 06 18:18:21.983820 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:18:21.983746    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cc40fe47-16bf-4a41-90e6-29f7e89f687f-registry-tls podName:cc40fe47-16bf-4a41-90e6-29f7e89f687f nodeName:}" failed. No retries permitted until 2026-07-06 18:18:23.983734289 +0000 UTC m=+39.141132753 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/cc40fe47-16bf-4a41-90e6-29f7e89f687f-registry-tls") pod "image-registry-58c4895898-bt6wr" (UID: "cc40fe47-16bf-4a41-90e6-29f7e89f687f") : secret "image-registry-tls" not found
Jul 06 18:18:21.983820 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:18:21.983765    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f354e957-d479-41ba-bf43-24303d500049-insights-runtime-extractor-tls podName:f354e957-d479-41ba-bf43-24303d500049 nodeName:}" failed. No retries permitted until 2026-07-06 18:18:23.983756314 +0000 UTC m=+39.141154780 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f354e957-d479-41ba-bf43-24303d500049-insights-runtime-extractor-tls") pod "insights-runtime-extractor-89h8f" (UID: "f354e957-d479-41ba-bf43-24303d500049") : secret "insights-runtime-extractor-tls" not found
Jul 06 18:18:22.061157 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:22.061116    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-758f599c5-4qf9q"]
Jul 06 18:18:22.080553 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:22.080522    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-758f599c5-4qf9q"]
Jul 06 18:18:22.080701 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:22.080595    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-758f599c5-4qf9q"
Jul 06 18:18:22.083551 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:22.083382    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-zjc9f\""
Jul 06 18:18:22.083551 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:22.083382    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Jul 06 18:18:22.083755 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:22.083559    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Jul 06 18:18:22.084576 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:22.084550    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Jul 06 18:18:22.084827 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:22.084807    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Jul 06 18:18:22.185676 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:22.185643    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/6bb15897-24f4-45fd-a6ce-9f608d5f86fc-signing-cabundle\") pod \"service-ca-758f599c5-4qf9q\" (UID: \"6bb15897-24f4-45fd-a6ce-9f608d5f86fc\") " pod="openshift-service-ca/service-ca-758f599c5-4qf9q"
Jul 06 18:18:22.185676 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:22.185677    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7whf2\" (UniqueName: \"kubernetes.io/projected/6bb15897-24f4-45fd-a6ce-9f608d5f86fc-kube-api-access-7whf2\") pod \"service-ca-758f599c5-4qf9q\" (UID: \"6bb15897-24f4-45fd-a6ce-9f608d5f86fc\") " pod="openshift-service-ca/service-ca-758f599c5-4qf9q"
Jul 06 18:18:22.185857 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:22.185767    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/6bb15897-24f4-45fd-a6ce-9f608d5f86fc-signing-key\") pod \"service-ca-758f599c5-4qf9q\" (UID: \"6bb15897-24f4-45fd-a6ce-9f608d5f86fc\") " pod="openshift-service-ca/service-ca-758f599c5-4qf9q"
Jul 06 18:18:22.286824 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:22.286748    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/6bb15897-24f4-45fd-a6ce-9f608d5f86fc-signing-cabundle\") pod \"service-ca-758f599c5-4qf9q\" (UID: \"6bb15897-24f4-45fd-a6ce-9f608d5f86fc\") " pod="openshift-service-ca/service-ca-758f599c5-4qf9q"
Jul 06 18:18:22.286824 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:22.286785    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7whf2\" (UniqueName: \"kubernetes.io/projected/6bb15897-24f4-45fd-a6ce-9f608d5f86fc-kube-api-access-7whf2\") pod \"service-ca-758f599c5-4qf9q\" (UID: \"6bb15897-24f4-45fd-a6ce-9f608d5f86fc\") " pod="openshift-service-ca/service-ca-758f599c5-4qf9q"
Jul 06 18:18:22.287062 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:22.286851    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/6bb15897-24f4-45fd-a6ce-9f608d5f86fc-signing-key\") pod \"service-ca-758f599c5-4qf9q\" (UID: \"6bb15897-24f4-45fd-a6ce-9f608d5f86fc\") " pod="openshift-service-ca/service-ca-758f599c5-4qf9q"
Jul 06 18:18:22.287594 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:22.287568    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/6bb15897-24f4-45fd-a6ce-9f608d5f86fc-signing-cabundle\") pod \"service-ca-758f599c5-4qf9q\" (UID: \"6bb15897-24f4-45fd-a6ce-9f608d5f86fc\") " pod="openshift-service-ca/service-ca-758f599c5-4qf9q"
Jul 06 18:18:22.289839 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:22.289815    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/6bb15897-24f4-45fd-a6ce-9f608d5f86fc-signing-key\") pod \"service-ca-758f599c5-4qf9q\" (UID: \"6bb15897-24f4-45fd-a6ce-9f608d5f86fc\") " pod="openshift-service-ca/service-ca-758f599c5-4qf9q"
Jul 06 18:18:22.293448 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:22.293423    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-kcr4l_c2ef9254-5ac3-45db-99a0-b5aff2a25d58/node-ca/0.log"
Jul 06 18:18:22.297403 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:22.297382    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7whf2\" (UniqueName: \"kubernetes.io/projected/6bb15897-24f4-45fd-a6ce-9f608d5f86fc-kube-api-access-7whf2\") pod \"service-ca-758f599c5-4qf9q\" (UID: \"6bb15897-24f4-45fd-a6ce-9f608d5f86fc\") " pod="openshift-service-ca/service-ca-758f599c5-4qf9q"
Jul 06 18:18:22.392053 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:22.392003    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-758f599c5-4qf9q"
Jul 06 18:18:22.557462 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:22.557275    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-758f599c5-4qf9q"]
Jul 06 18:18:22.560954 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:18:22.560923    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6bb15897_24f4_45fd_a6ce_9f608d5f86fc.slice/crio-1be910600239b935ef40a7d2d9ed2515743f5d28310907461e25a002eed1b30e WatchSource:0}: Error finding container 1be910600239b935ef40a7d2d9ed2515743f5d28310907461e25a002eed1b30e: Status 404 returned error can't find the container with id 1be910600239b935ef40a7d2d9ed2515743f5d28310907461e25a002eed1b30e
Jul 06 18:18:22.589503 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:22.589434    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-758f599c5-4qf9q" event={"ID":"6bb15897-24f4-45fd-a6ce-9f608d5f86fc","Type":"ContainerStarted","Data":"1be910600239b935ef40a7d2d9ed2515743f5d28310907461e25a002eed1b30e"}
Jul 06 18:18:24.004917 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:24.004088    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/24fc32df-2fce-41bf-8a8d-849b392bcb8f-metrics-tls\") pod \"dns-default-lzzjn\" (UID: \"24fc32df-2fce-41bf-8a8d-849b392bcb8f\") " pod="openshift-dns/dns-default-lzzjn"
Jul 06 18:18:24.004917 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:24.004143    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f354e957-d479-41ba-bf43-24303d500049-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-89h8f\" (UID: \"f354e957-d479-41ba-bf43-24303d500049\") " pod="openshift-insights/insights-runtime-extractor-89h8f"
Jul 06 18:18:24.004917 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:24.004216    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f354e957-d479-41ba-bf43-24303d500049-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-89h8f\" (UID: \"f354e957-d479-41ba-bf43-24303d500049\") " pod="openshift-insights/insights-runtime-extractor-89h8f"
Jul 06 18:18:24.004917 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:24.004283    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cc40fe47-16bf-4a41-90e6-29f7e89f687f-registry-tls\") pod \"image-registry-58c4895898-bt6wr\" (UID: \"cc40fe47-16bf-4a41-90e6-29f7e89f687f\") " pod="openshift-image-registry/image-registry-58c4895898-bt6wr"
Jul 06 18:18:24.004917 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:18:24.004441    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 06 18:18:24.004917 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:18:24.004455    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-58c4895898-bt6wr: secret "image-registry-tls" not found
Jul 06 18:18:24.004917 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:18:24.004514    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cc40fe47-16bf-4a41-90e6-29f7e89f687f-registry-tls podName:cc40fe47-16bf-4a41-90e6-29f7e89f687f nodeName:}" failed. No retries permitted until 2026-07-06 18:18:28.004495936 +0000 UTC m=+43.161894423 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/cc40fe47-16bf-4a41-90e6-29f7e89f687f-registry-tls") pod "image-registry-58c4895898-bt6wr" (UID: "cc40fe47-16bf-4a41-90e6-29f7e89f687f") : secret "image-registry-tls" not found
Jul 06 18:18:24.004917 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:18:24.004745    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 06 18:18:24.004917 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:18:24.004799    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f354e957-d479-41ba-bf43-24303d500049-insights-runtime-extractor-tls podName:f354e957-d479-41ba-bf43-24303d500049 nodeName:}" failed. No retries permitted until 2026-07-06 18:18:28.004781278 +0000 UTC m=+43.162179754 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f354e957-d479-41ba-bf43-24303d500049-insights-runtime-extractor-tls") pod "insights-runtime-extractor-89h8f" (UID: "f354e957-d479-41ba-bf43-24303d500049") : secret "insights-runtime-extractor-tls" not found
Jul 06 18:18:24.004917 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:18:24.004854    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 06 18:18:24.004917 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:18:24.004889    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24fc32df-2fce-41bf-8a8d-849b392bcb8f-metrics-tls podName:24fc32df-2fce-41bf-8a8d-849b392bcb8f nodeName:}" failed. No retries permitted until 2026-07-06 18:18:28.004875561 +0000 UTC m=+43.162274036 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/24fc32df-2fce-41bf-8a8d-849b392bcb8f-metrics-tls") pod "dns-default-lzzjn" (UID: "24fc32df-2fce-41bf-8a8d-849b392bcb8f") : secret "dns-default-metrics-tls" not found
Jul 06 18:18:24.004917 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:18:24.004887    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 06 18:18:24.004917 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:18:24.004920    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f354e957-d479-41ba-bf43-24303d500049-kube-rbac-proxy-cm podName:f354e957-d479-41ba-bf43-24303d500049 nodeName:}" failed. No retries permitted until 2026-07-06 18:18:28.004912105 +0000 UTC m=+43.162310569 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f354e957-d479-41ba-bf43-24303d500049-kube-rbac-proxy-cm") pod "insights-runtime-extractor-89h8f" (UID: "f354e957-d479-41ba-bf43-24303d500049") : configmap "kube-rbac-proxy" not found
Jul 06 18:18:27.605220 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:27.605183    2580 generic.go:356] "Generic (PLEG): container finished" podID="a420108c-26d7-46d2-a3d5-3b354aa01af0" containerID="8b002ab4cb35a8987e8390c744c054ab06e5f272310f23d59f18dcecf04a5db4" exitCode=0
Jul 06 18:18:27.605728 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:27.605273    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-sl8sj" event={"ID":"a420108c-26d7-46d2-a3d5-3b354aa01af0","Type":"ContainerDied","Data":"8b002ab4cb35a8987e8390c744c054ab06e5f272310f23d59f18dcecf04a5db4"}
Jul 06 18:18:27.606760 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:27.606736    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-nmspz" event={"ID":"534f2c50-cd67-4d03-90b9-5076223138ff","Type":"ContainerStarted","Data":"c3a2ac46c8e4a8d6a80d25828ffaa6d88eb77e15d2eec9c4657b9cdc7ee58018"}
Jul 06 18:18:27.608201 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:27.608174    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-758f599c5-4qf9q" event={"ID":"6bb15897-24f4-45fd-a6ce-9f608d5f86fc","Type":"ContainerStarted","Data":"d5926226451c9122c0e7a5b50c3198851bf75104eef051b91fe03b6e55e875a1"}
Jul 06 18:18:27.645132 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:27.645069    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-758f599c5-4qf9q" podStartSLOduration=1.181238738 podStartE2EDuration="5.645045826s" totalImagesPullingTime="4.463807088s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:18:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:18:27.644490527 +0000 UTC m=+42.801889014" watchObservedRunningTime="2026-07-06 18:18:27.645045826 +0000 UTC m=+42.802444293"
Jul 06 18:18:27.665322 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:27.662478    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-nmspz" podStartSLOduration=1.38258215 podStartE2EDuration="7.662457994s" totalImagesPullingTime="6.279875844s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:18:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:18:27.66116232 +0000 UTC m=+42.818560808" watchObservedRunningTime="2026-07-06 18:18:27.662457994 +0000 UTC m=+42.819856486"
Jul 06 18:18:28.038044 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:28.037921    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f354e957-d479-41ba-bf43-24303d500049-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-89h8f\" (UID: \"f354e957-d479-41ba-bf43-24303d500049\") " pod="openshift-insights/insights-runtime-extractor-89h8f"
Jul 06 18:18:28.038044 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:28.037987    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cc40fe47-16bf-4a41-90e6-29f7e89f687f-registry-tls\") pod \"image-registry-58c4895898-bt6wr\" (UID: \"cc40fe47-16bf-4a41-90e6-29f7e89f687f\") " pod="openshift-image-registry/image-registry-58c4895898-bt6wr"
Jul 06 18:18:28.038044 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:28.038030    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/24fc32df-2fce-41bf-8a8d-849b392bcb8f-metrics-tls\") pod \"dns-default-lzzjn\" (UID: \"24fc32df-2fce-41bf-8a8d-849b392bcb8f\") " pod="openshift-dns/dns-default-lzzjn"
Jul 06 18:18:28.038322 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:28.038053    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f354e957-d479-41ba-bf43-24303d500049-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-89h8f\" (UID: \"f354e957-d479-41ba-bf43-24303d500049\") " pod="openshift-insights/insights-runtime-extractor-89h8f"
Jul 06 18:18:28.038322 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:18:28.038077    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 06 18:18:28.038322 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:18:28.038139    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 06 18:18:28.038322 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:18:28.038154    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 06 18:18:28.038322 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:18:28.038166    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f354e957-d479-41ba-bf43-24303d500049-kube-rbac-proxy-cm podName:f354e957-d479-41ba-bf43-24303d500049 nodeName:}" failed. No retries permitted until 2026-07-06 18:18:36.038145711 +0000 UTC m=+51.195544198 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f354e957-d479-41ba-bf43-24303d500049-kube-rbac-proxy-cm") pod "insights-runtime-extractor-89h8f" (UID: "f354e957-d479-41ba-bf43-24303d500049") : configmap "kube-rbac-proxy" not found
Jul 06 18:18:28.038322 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:18:28.038246    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24fc32df-2fce-41bf-8a8d-849b392bcb8f-metrics-tls podName:24fc32df-2fce-41bf-8a8d-849b392bcb8f nodeName:}" failed. No retries permitted until 2026-07-06 18:18:36.0382329 +0000 UTC m=+51.195631379 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/24fc32df-2fce-41bf-8a8d-849b392bcb8f-metrics-tls") pod "dns-default-lzzjn" (UID: "24fc32df-2fce-41bf-8a8d-849b392bcb8f") : secret "dns-default-metrics-tls" not found
Jul 06 18:18:28.038322 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:18:28.038154    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 06 18:18:28.038322 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:18:28.038156    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-58c4895898-bt6wr: secret "image-registry-tls" not found
Jul 06 18:18:28.038322 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:18:28.038288    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f354e957-d479-41ba-bf43-24303d500049-insights-runtime-extractor-tls podName:f354e957-d479-41ba-bf43-24303d500049 nodeName:}" failed. No retries permitted until 2026-07-06 18:18:36.038278489 +0000 UTC m=+51.195676967 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f354e957-d479-41ba-bf43-24303d500049-insights-runtime-extractor-tls") pod "insights-runtime-extractor-89h8f" (UID: "f354e957-d479-41ba-bf43-24303d500049") : secret "insights-runtime-extractor-tls" not found
Jul 06 18:18:28.038803 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:18:28.038346    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cc40fe47-16bf-4a41-90e6-29f7e89f687f-registry-tls podName:cc40fe47-16bf-4a41-90e6-29f7e89f687f nodeName:}" failed. No retries permitted until 2026-07-06 18:18:36.038331307 +0000 UTC m=+51.195729773 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/cc40fe47-16bf-4a41-90e6-29f7e89f687f-registry-tls") pod "image-registry-58c4895898-bt6wr" (UID: "cc40fe47-16bf-4a41-90e6-29f7e89f687f") : secret "image-registry-tls" not found
Jul 06 18:18:28.614768 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:28.614736    2580 generic.go:356] "Generic (PLEG): container finished" podID="a420108c-26d7-46d2-a3d5-3b354aa01af0" containerID="1c458a3e3883a01f2cf29f27de29c4c183f148831ac4d2e944d2a782b46c5770" exitCode=0
Jul 06 18:18:28.615268 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:28.614850    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-sl8sj" event={"ID":"a420108c-26d7-46d2-a3d5-3b354aa01af0","Type":"ContainerDied","Data":"1c458a3e3883a01f2cf29f27de29c4c183f148831ac4d2e944d2a782b46c5770"}
Jul 06 18:18:29.620577 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:29.620541    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-sl8sj" event={"ID":"a420108c-26d7-46d2-a3d5-3b354aa01af0","Type":"ContainerStarted","Data":"52392317607f8ddba79c3976b3d06c232c690b593852a2c0ad4ba9eb76192649"}
Jul 06 18:18:29.645198 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:29.645134    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-sl8sj" podStartSLOduration=14.795058161 podStartE2EDuration="44.64510435s" totalImagesPullingTime="29.850046189s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 18:17:45 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:18:29.64294526 +0000 UTC m=+44.800343745" watchObservedRunningTime="2026-07-06 18:18:29.64510435 +0000 UTC m=+44.802502838"
Jul 06 18:18:36.109781 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:36.109738    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cc40fe47-16bf-4a41-90e6-29f7e89f687f-registry-tls\") pod \"image-registry-58c4895898-bt6wr\" (UID: \"cc40fe47-16bf-4a41-90e6-29f7e89f687f\") " pod="openshift-image-registry/image-registry-58c4895898-bt6wr"
Jul 06 18:18:36.110296 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:36.109793    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/24fc32df-2fce-41bf-8a8d-849b392bcb8f-metrics-tls\") pod \"dns-default-lzzjn\" (UID: \"24fc32df-2fce-41bf-8a8d-849b392bcb8f\") " pod="openshift-dns/dns-default-lzzjn"
Jul 06 18:18:36.110296 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:36.109825    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f354e957-d479-41ba-bf43-24303d500049-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-89h8f\" (UID: \"f354e957-d479-41ba-bf43-24303d500049\") " pod="openshift-insights/insights-runtime-extractor-89h8f"
Jul 06 18:18:36.110296 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:36.109892    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f354e957-d479-41ba-bf43-24303d500049-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-89h8f\" (UID: \"f354e957-d479-41ba-bf43-24303d500049\") " pod="openshift-insights/insights-runtime-extractor-89h8f"
Jul 06 18:18:36.110296 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:18:36.109980    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 06 18:18:36.110296 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:18:36.110079    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f354e957-d479-41ba-bf43-24303d500049-insights-runtime-extractor-tls podName:f354e957-d479-41ba-bf43-24303d500049 nodeName:}" failed. No retries permitted until 2026-07-06 18:18:52.110061469 +0000 UTC m=+67.267459933 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f354e957-d479-41ba-bf43-24303d500049-insights-runtime-extractor-tls") pod "insights-runtime-extractor-89h8f" (UID: "f354e957-d479-41ba-bf43-24303d500049") : secret "insights-runtime-extractor-tls" not found
Jul 06 18:18:36.110575 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:36.110541    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f354e957-d479-41ba-bf43-24303d500049-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-89h8f\" (UID: \"f354e957-d479-41ba-bf43-24303d500049\") " pod="openshift-insights/insights-runtime-extractor-89h8f"
Jul 06 18:18:36.112758 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:36.112735    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/24fc32df-2fce-41bf-8a8d-849b392bcb8f-metrics-tls\") pod \"dns-default-lzzjn\" (UID: \"24fc32df-2fce-41bf-8a8d-849b392bcb8f\") " pod="openshift-dns/dns-default-lzzjn"
Jul 06 18:18:36.112863 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:36.112771    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cc40fe47-16bf-4a41-90e6-29f7e89f687f-registry-tls\") pod \"image-registry-58c4895898-bt6wr\" (UID: \"cc40fe47-16bf-4a41-90e6-29f7e89f687f\") " pod="openshift-image-registry/image-registry-58c4895898-bt6wr"
Jul 06 18:18:36.126412 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:36.126383    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-58c4895898-bt6wr"
Jul 06 18:18:36.179422 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:36.179384    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-lzzjn"
Jul 06 18:18:36.283811 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:36.283779    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-58c4895898-bt6wr"]
Jul 06 18:18:36.288376 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:18:36.288339    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc40fe47_16bf_4a41_90e6_29f7e89f687f.slice/crio-81848c3c59e98869f705fe9b7f27e18a6fc13a602f85ce41b5503c468f37faac WatchSource:0}: Error finding container 81848c3c59e98869f705fe9b7f27e18a6fc13a602f85ce41b5503c468f37faac: Status 404 returned error can't find the container with id 81848c3c59e98869f705fe9b7f27e18a6fc13a602f85ce41b5503c468f37faac
Jul 06 18:18:36.341003 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:36.340975    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-lzzjn"]
Jul 06 18:18:36.344734 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:18:36.344707    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24fc32df_2fce_41bf_8a8d_849b392bcb8f.slice/crio-827402dd112fa3a2aca36966276f0361257f45af1099e593c68f63e4d478e916 WatchSource:0}: Error finding container 827402dd112fa3a2aca36966276f0361257f45af1099e593c68f63e4d478e916: Status 404 returned error can't find the container with id 827402dd112fa3a2aca36966276f0361257f45af1099e593c68f63e4d478e916
Jul 06 18:18:36.638209 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:36.638107    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-58c4895898-bt6wr" event={"ID":"cc40fe47-16bf-4a41-90e6-29f7e89f687f","Type":"ContainerStarted","Data":"c91a73467725d2da41229740c9e90c20a4024e07c60f6862d54f7389fee5f844"}
Jul 06 18:18:36.638209 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:36.638172    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-58c4895898-bt6wr" event={"ID":"cc40fe47-16bf-4a41-90e6-29f7e89f687f","Type":"ContainerStarted","Data":"81848c3c59e98869f705fe9b7f27e18a6fc13a602f85ce41b5503c468f37faac"}
Jul 06 18:18:36.638410 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:36.638240    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-58c4895898-bt6wr"
Jul 06 18:18:36.639329 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:36.639307    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-lzzjn" event={"ID":"24fc32df-2fce-41bf-8a8d-849b392bcb8f","Type":"ContainerStarted","Data":"827402dd112fa3a2aca36966276f0361257f45af1099e593c68f63e4d478e916"}
Jul 06 18:18:38.646157 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:38.646123    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-lzzjn" event={"ID":"24fc32df-2fce-41bf-8a8d-849b392bcb8f","Type":"ContainerStarted","Data":"fc4f11b14387126d120cefff27b6b7870e4b4ebbde1a183f3520059b0e871735"}
Jul 06 18:18:38.646553 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:38.646177    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-lzzjn" event={"ID":"24fc32df-2fce-41bf-8a8d-849b392bcb8f","Type":"ContainerStarted","Data":"b18265c8db9ecdb0b2829ac57efd0a6fe98af1dd6bec9399020db9b65a12bdd5"}
Jul 06 18:18:38.646553 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:38.646252    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-lzzjn"
Jul 06 18:18:38.663764 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:38.663708    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-lzzjn" podStartSLOduration=17.062428551 podStartE2EDuration="18.663692055s" totalImagesPullingTime="1.601263504s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:18:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:18:38.663390074 +0000 UTC m=+53.820788569" watchObservedRunningTime="2026-07-06 18:18:38.663692055 +0000 UTC m=+53.821090540"
Jul 06 18:18:38.664570 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:38.664536    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-58c4895898-bt6wr" podStartSLOduration=57.664520887 podStartE2EDuration="57.664520887s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:17:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:18:36.660297236 +0000 UTC m=+51.817695721" watchObservedRunningTime="2026-07-06 18:18:38.664520887 +0000 UTC m=+53.821919373"
Jul 06 18:18:41.580377 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:41.580344    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7d5956586-ktr6l"]
Jul 06 18:18:41.583053 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:41.583031    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-5f4668c794-qxs4w"]
Jul 06 18:18:41.583265 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:41.583136    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7d5956586-ktr6l"
Jul 06 18:18:41.584877 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:41.584858    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-744848fcbc-zc4nm"]
Jul 06 18:18:41.584980 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:41.584964    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5f4668c794-qxs4w"
Jul 06 18:18:41.586802 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:41.586786    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-744848fcbc-zc4nm"
Jul 06 18:18:41.586976 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:41.586956    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 06 18:18:41.587468 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:41.587449    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-cq8ln\""
Jul 06 18:18:41.587544 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:41.587472    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 06 18:18:41.588561 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:41.588545    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Jul 06 18:18:41.588652 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:41.588591    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Jul 06 18:18:41.588652 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:41.588622    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 06 18:18:41.591619 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:41.591599    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Jul 06 18:18:41.591717 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:41.591638    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Jul 06 18:18:41.591717 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:41.591645    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Jul 06 18:18:41.591802 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:41.591726    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Jul 06 18:18:41.604408 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:41.604384    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-5f4668c794-qxs4w"]
Jul 06 18:18:41.605494 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:41.605462    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-744848fcbc-zc4nm"]
Jul 06 18:18:41.606291 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:41.606270    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7d5956586-ktr6l"]
Jul 06 18:18:41.646891 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:41.646863    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-58c4895898-bt6wr"]
Jul 06 18:18:41.654310 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:41.654286    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/dd383d1b-2e82-407d-84f5-09f902c288e1-additional-service-ca\") pod \"cluster-proxy-proxy-agent-744848fcbc-zc4nm\" (UID: \"dd383d1b-2e82-407d-84f5-09f902c288e1\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-744848fcbc-zc4nm"
Jul 06 18:18:41.654452 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:41.654331    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/dd383d1b-2e82-407d-84f5-09f902c288e1-hub\") pod \"cluster-proxy-proxy-agent-744848fcbc-zc4nm\" (UID: \"dd383d1b-2e82-407d-84f5-09f902c288e1\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-744848fcbc-zc4nm"
Jul 06 18:18:41.654452 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:41.654351    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6nrrs\" (UniqueName: \"kubernetes.io/projected/dd383d1b-2e82-407d-84f5-09f902c288e1-kube-api-access-6nrrs\") pod \"cluster-proxy-proxy-agent-744848fcbc-zc4nm\" (UID: \"dd383d1b-2e82-407d-84f5-09f902c288e1\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-744848fcbc-zc4nm"
Jul 06 18:18:41.654452 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:41.654367    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/672e2368-ae59-4ab7-be3f-439daa363e9d-tmp\") pod \"klusterlet-addon-workmgr-5f4668c794-qxs4w\" (UID: \"672e2368-ae59-4ab7-be3f-439daa363e9d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5f4668c794-qxs4w"
Jul 06 18:18:41.654452 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:41.654381    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/672e2368-ae59-4ab7-be3f-439daa363e9d-klusterlet-config\") pod \"klusterlet-addon-workmgr-5f4668c794-qxs4w\" (UID: \"672e2368-ae59-4ab7-be3f-439daa363e9d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5f4668c794-qxs4w"
Jul 06 18:18:41.654452 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:41.654423    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/dfe77255-7abf-49b6-acc4-0846089ad14e-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7d5956586-ktr6l\" (UID: \"dfe77255-7abf-49b6-acc4-0846089ad14e\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7d5956586-ktr6l"
Jul 06 18:18:41.654452 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:41.654442    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/dd383d1b-2e82-407d-84f5-09f902c288e1-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-744848fcbc-zc4nm\" (UID: \"dd383d1b-2e82-407d-84f5-09f902c288e1\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-744848fcbc-zc4nm"
Jul 06 18:18:41.654795 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:41.654487    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/dd383d1b-2e82-407d-84f5-09f902c288e1-ca\") pod \"cluster-proxy-proxy-agent-744848fcbc-zc4nm\" (UID: \"dd383d1b-2e82-407d-84f5-09f902c288e1\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-744848fcbc-zc4nm"
Jul 06 18:18:41.654795 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:41.654549    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j22pf\" (UniqueName: \"kubernetes.io/projected/672e2368-ae59-4ab7-be3f-439daa363e9d-kube-api-access-j22pf\") pod \"klusterlet-addon-workmgr-5f4668c794-qxs4w\" (UID: \"672e2368-ae59-4ab7-be3f-439daa363e9d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5f4668c794-qxs4w"
Jul 06 18:18:41.654795 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:41.654584    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/dd383d1b-2e82-407d-84f5-09f902c288e1-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-744848fcbc-zc4nm\" (UID: \"dd383d1b-2e82-407d-84f5-09f902c288e1\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-744848fcbc-zc4nm"
Jul 06 18:18:41.654795 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:41.654628    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t2smn\" (UniqueName: \"kubernetes.io/projected/dfe77255-7abf-49b6-acc4-0846089ad14e-kube-api-access-t2smn\") pod \"managed-serviceaccount-addon-agent-7d5956586-ktr6l\" (UID: \"dfe77255-7abf-49b6-acc4-0846089ad14e\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7d5956586-ktr6l"
Jul 06 18:18:41.698958 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:41.698918    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-757fd97788-t78bl"]
Jul 06 18:18:41.700900 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:41.700878    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-757fd97788-t78bl"
Jul 06 18:18:41.718835 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:41.718810    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-757fd97788-t78bl"]
Jul 06 18:18:41.755853 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:41.755811    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/128b0f11-7a7d-44eb-9830-5208fb456e64-bound-sa-token\") pod \"image-registry-757fd97788-t78bl\" (UID: \"128b0f11-7a7d-44eb-9830-5208fb456e64\") " pod="openshift-image-registry/image-registry-757fd97788-t78bl"
Jul 06 18:18:41.756059 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:41.755886    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/128b0f11-7a7d-44eb-9830-5208fb456e64-image-registry-private-configuration\") pod \"image-registry-757fd97788-t78bl\" (UID: \"128b0f11-7a7d-44eb-9830-5208fb456e64\") " pod="openshift-image-registry/image-registry-757fd97788-t78bl"
Jul 06 18:18:41.756059 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:41.755915    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/dd383d1b-2e82-407d-84f5-09f902c288e1-hub\") pod \"cluster-proxy-proxy-agent-744848fcbc-zc4nm\" (UID: \"dd383d1b-2e82-407d-84f5-09f902c288e1\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-744848fcbc-zc4nm"
Jul 06 18:18:41.756059 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:41.755938    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r4r5s\" (UniqueName: \"kubernetes.io/projected/128b0f11-7a7d-44eb-9830-5208fb456e64-kube-api-access-r4r5s\") pod \"image-registry-757fd97788-t78bl\" (UID: \"128b0f11-7a7d-44eb-9830-5208fb456e64\") " pod="openshift-image-registry/image-registry-757fd97788-t78bl"
Jul 06 18:18:41.756059 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:41.755971    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6nrrs\" (UniqueName: \"kubernetes.io/projected/dd383d1b-2e82-407d-84f5-09f902c288e1-kube-api-access-6nrrs\") pod \"cluster-proxy-proxy-agent-744848fcbc-zc4nm\" (UID: \"dd383d1b-2e82-407d-84f5-09f902c288e1\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-744848fcbc-zc4nm"
Jul 06 18:18:41.756059 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:41.755994    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/672e2368-ae59-4ab7-be3f-439daa363e9d-tmp\") pod \"klusterlet-addon-workmgr-5f4668c794-qxs4w\" (UID: \"672e2368-ae59-4ab7-be3f-439daa363e9d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5f4668c794-qxs4w"
Jul 06 18:18:41.756059 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:41.756030    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/672e2368-ae59-4ab7-be3f-439daa363e9d-klusterlet-config\") pod \"klusterlet-addon-workmgr-5f4668c794-qxs4w\" (UID: \"672e2368-ae59-4ab7-be3f-439daa363e9d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5f4668c794-qxs4w"
Jul 06 18:18:41.756351 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:41.756102    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/dfe77255-7abf-49b6-acc4-0846089ad14e-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7d5956586-ktr6l\" (UID: \"dfe77255-7abf-49b6-acc4-0846089ad14e\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7d5956586-ktr6l"
Jul 06 18:18:41.756351 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:41.756130    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/128b0f11-7a7d-44eb-9830-5208fb456e64-installation-pull-secrets\") pod \"image-registry-757fd97788-t78bl\" (UID: \"128b0f11-7a7d-44eb-9830-5208fb456e64\") " pod="openshift-image-registry/image-registry-757fd97788-t78bl"
Jul 06 18:18:41.756351 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:41.756158    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/128b0f11-7a7d-44eb-9830-5208fb456e64-registry-tls\") pod \"image-registry-757fd97788-t78bl\" (UID: \"128b0f11-7a7d-44eb-9830-5208fb456e64\") " pod="openshift-image-registry/image-registry-757fd97788-t78bl"
Jul 06 18:18:41.756351 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:41.756221    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/dd383d1b-2e82-407d-84f5-09f902c288e1-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-744848fcbc-zc4nm\" (UID: \"dd383d1b-2e82-407d-84f5-09f902c288e1\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-744848fcbc-zc4nm"
Jul 06 18:18:41.756351 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:41.756258    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/dd383d1b-2e82-407d-84f5-09f902c288e1-ca\") pod \"cluster-proxy-proxy-agent-744848fcbc-zc4nm\" (UID: \"dd383d1b-2e82-407d-84f5-09f902c288e1\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-744848fcbc-zc4nm"
Jul 06 18:18:41.756351 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:41.756299    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j22pf\" (UniqueName: \"kubernetes.io/projected/672e2368-ae59-4ab7-be3f-439daa363e9d-kube-api-access-j22pf\") pod \"klusterlet-addon-workmgr-5f4668c794-qxs4w\" (UID: \"672e2368-ae59-4ab7-be3f-439daa363e9d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5f4668c794-qxs4w"
Jul 06 18:18:41.756652 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:41.756441    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/672e2368-ae59-4ab7-be3f-439daa363e9d-tmp\") pod \"klusterlet-addon-workmgr-5f4668c794-qxs4w\" (UID: \"672e2368-ae59-4ab7-be3f-439daa363e9d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5f4668c794-qxs4w"
Jul 06 18:18:41.756652 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:41.756490    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/dd383d1b-2e82-407d-84f5-09f902c288e1-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-744848fcbc-zc4nm\" (UID: \"dd383d1b-2e82-407d-84f5-09f902c288e1\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-744848fcbc-zc4nm"
Jul 06 18:18:41.756652 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:41.756541    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t2smn\" (UniqueName: \"kubernetes.io/projected/dfe77255-7abf-49b6-acc4-0846089ad14e-kube-api-access-t2smn\") pod \"managed-serviceaccount-addon-agent-7d5956586-ktr6l\" (UID: \"dfe77255-7abf-49b6-acc4-0846089ad14e\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7d5956586-ktr6l"
Jul 06 18:18:41.756652 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:41.756586    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/128b0f11-7a7d-44eb-9830-5208fb456e64-ca-trust-extracted\") pod \"image-registry-757fd97788-t78bl\" (UID: \"128b0f11-7a7d-44eb-9830-5208fb456e64\") " pod="openshift-image-registry/image-registry-757fd97788-t78bl"
Jul 06 18:18:41.756652 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:41.756610    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/128b0f11-7a7d-44eb-9830-5208fb456e64-registry-certificates\") pod \"image-registry-757fd97788-t78bl\" (UID: \"128b0f11-7a7d-44eb-9830-5208fb456e64\") " pod="openshift-image-registry/image-registry-757fd97788-t78bl"
Jul 06 18:18:41.756652 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:41.756632    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/128b0f11-7a7d-44eb-9830-5208fb456e64-trusted-ca\") pod \"image-registry-757fd97788-t78bl\" (UID: \"128b0f11-7a7d-44eb-9830-5208fb456e64\") " pod="openshift-image-registry/image-registry-757fd97788-t78bl"
Jul 06 18:18:41.756950 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:41.756656    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/dd383d1b-2e82-407d-84f5-09f902c288e1-additional-service-ca\") pod \"cluster-proxy-proxy-agent-744848fcbc-zc4nm\" (UID: \"dd383d1b-2e82-407d-84f5-09f902c288e1\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-744848fcbc-zc4nm"
Jul 06 18:18:41.757392 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:41.757367    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/dd383d1b-2e82-407d-84f5-09f902c288e1-additional-service-ca\") pod \"cluster-proxy-proxy-agent-744848fcbc-zc4nm\" (UID: \"dd383d1b-2e82-407d-84f5-09f902c288e1\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-744848fcbc-zc4nm"
Jul 06 18:18:41.758933 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:41.758882    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/dd383d1b-2e82-407d-84f5-09f902c288e1-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-744848fcbc-zc4nm\" (UID: \"dd383d1b-2e82-407d-84f5-09f902c288e1\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-744848fcbc-zc4nm"
Jul 06 18:18:41.759053 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:41.758961    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/672e2368-ae59-4ab7-be3f-439daa363e9d-klusterlet-config\") pod \"klusterlet-addon-workmgr-5f4668c794-qxs4w\" (UID: \"672e2368-ae59-4ab7-be3f-439daa363e9d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5f4668c794-qxs4w"
Jul 06 18:18:41.759171 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:41.759090    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/dd383d1b-2e82-407d-84f5-09f902c288e1-hub\") pod \"cluster-proxy-proxy-agent-744848fcbc-zc4nm\" (UID: \"dd383d1b-2e82-407d-84f5-09f902c288e1\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-744848fcbc-zc4nm"
Jul 06 18:18:41.759171 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:41.759134    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/dfe77255-7abf-49b6-acc4-0846089ad14e-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7d5956586-ktr6l\" (UID: \"dfe77255-7abf-49b6-acc4-0846089ad14e\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7d5956586-ktr6l"
Jul 06 18:18:41.759171 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:41.759166    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/dd383d1b-2e82-407d-84f5-09f902c288e1-ca\") pod \"cluster-proxy-proxy-agent-744848fcbc-zc4nm\" (UID: \"dd383d1b-2e82-407d-84f5-09f902c288e1\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-744848fcbc-zc4nm"
Jul 06 18:18:41.759325 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:41.759306    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/dd383d1b-2e82-407d-84f5-09f902c288e1-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-744848fcbc-zc4nm\" (UID: \"dd383d1b-2e82-407d-84f5-09f902c288e1\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-744848fcbc-zc4nm"
Jul 06 18:18:41.768786 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:41.768749    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6nrrs\" (UniqueName: \"kubernetes.io/projected/dd383d1b-2e82-407d-84f5-09f902c288e1-kube-api-access-6nrrs\") pod \"cluster-proxy-proxy-agent-744848fcbc-zc4nm\" (UID: \"dd383d1b-2e82-407d-84f5-09f902c288e1\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-744848fcbc-zc4nm"
Jul 06 18:18:41.770130 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:41.770111    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t2smn\" (UniqueName: \"kubernetes.io/projected/dfe77255-7abf-49b6-acc4-0846089ad14e-kube-api-access-t2smn\") pod \"managed-serviceaccount-addon-agent-7d5956586-ktr6l\" (UID: \"dfe77255-7abf-49b6-acc4-0846089ad14e\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7d5956586-ktr6l"
Jul 06 18:18:41.771101 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:41.771071    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j22pf\" (UniqueName: \"kubernetes.io/projected/672e2368-ae59-4ab7-be3f-439daa363e9d-kube-api-access-j22pf\") pod \"klusterlet-addon-workmgr-5f4668c794-qxs4w\" (UID: \"672e2368-ae59-4ab7-be3f-439daa363e9d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5f4668c794-qxs4w"
Jul 06 18:18:41.858057 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:41.857950    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/128b0f11-7a7d-44eb-9830-5208fb456e64-bound-sa-token\") pod \"image-registry-757fd97788-t78bl\" (UID: \"128b0f11-7a7d-44eb-9830-5208fb456e64\") " pod="openshift-image-registry/image-registry-757fd97788-t78bl"
Jul 06 18:18:41.858057 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:41.858001    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/128b0f11-7a7d-44eb-9830-5208fb456e64-image-registry-private-configuration\") pod \"image-registry-757fd97788-t78bl\" (UID: \"128b0f11-7a7d-44eb-9830-5208fb456e64\") " pod="openshift-image-registry/image-registry-757fd97788-t78bl"
Jul 06 18:18:41.858057 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:41.858049    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r4r5s\" (UniqueName: \"kubernetes.io/projected/128b0f11-7a7d-44eb-9830-5208fb456e64-kube-api-access-r4r5s\") pod \"image-registry-757fd97788-t78bl\" (UID: \"128b0f11-7a7d-44eb-9830-5208fb456e64\") " pod="openshift-image-registry/image-registry-757fd97788-t78bl"
Jul 06 18:18:41.858307 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:41.858101    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/128b0f11-7a7d-44eb-9830-5208fb456e64-installation-pull-secrets\") pod \"image-registry-757fd97788-t78bl\" (UID: \"128b0f11-7a7d-44eb-9830-5208fb456e64\") " pod="openshift-image-registry/image-registry-757fd97788-t78bl"
Jul 06 18:18:41.858307 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:41.858125    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/128b0f11-7a7d-44eb-9830-5208fb456e64-registry-tls\") pod \"image-registry-757fd97788-t78bl\" (UID: \"128b0f11-7a7d-44eb-9830-5208fb456e64\") " pod="openshift-image-registry/image-registry-757fd97788-t78bl"
Jul 06 18:18:41.858307 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:41.858163    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/128b0f11-7a7d-44eb-9830-5208fb456e64-ca-trust-extracted\") pod \"image-registry-757fd97788-t78bl\" (UID: \"128b0f11-7a7d-44eb-9830-5208fb456e64\") " pod="openshift-image-registry/image-registry-757fd97788-t78bl"
Jul 06 18:18:41.858307 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:41.858181    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/128b0f11-7a7d-44eb-9830-5208fb456e64-registry-certificates\") pod \"image-registry-757fd97788-t78bl\" (UID: \"128b0f11-7a7d-44eb-9830-5208fb456e64\") " pod="openshift-image-registry/image-registry-757fd97788-t78bl"
Jul 06 18:18:41.858307 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:41.858196    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/128b0f11-7a7d-44eb-9830-5208fb456e64-trusted-ca\") pod \"image-registry-757fd97788-t78bl\" (UID: \"128b0f11-7a7d-44eb-9830-5208fb456e64\") " pod="openshift-image-registry/image-registry-757fd97788-t78bl"
Jul 06 18:18:41.858793 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:41.858767    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/128b0f11-7a7d-44eb-9830-5208fb456e64-ca-trust-extracted\") pod \"image-registry-757fd97788-t78bl\" (UID: \"128b0f11-7a7d-44eb-9830-5208fb456e64\") " pod="openshift-image-registry/image-registry-757fd97788-t78bl"
Jul 06 18:18:41.859097 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:41.859076    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/128b0f11-7a7d-44eb-9830-5208fb456e64-registry-certificates\") pod \"image-registry-757fd97788-t78bl\" (UID: \"128b0f11-7a7d-44eb-9830-5208fb456e64\") " pod="openshift-image-registry/image-registry-757fd97788-t78bl"
Jul 06 18:18:41.859461 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:41.859438    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/128b0f11-7a7d-44eb-9830-5208fb456e64-trusted-ca\") pod \"image-registry-757fd97788-t78bl\" (UID: \"128b0f11-7a7d-44eb-9830-5208fb456e64\") " pod="openshift-image-registry/image-registry-757fd97788-t78bl"
Jul 06 18:18:41.860532 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:41.860509    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/128b0f11-7a7d-44eb-9830-5208fb456e64-installation-pull-secrets\") pod \"image-registry-757fd97788-t78bl\" (UID: \"128b0f11-7a7d-44eb-9830-5208fb456e64\") " pod="openshift-image-registry/image-registry-757fd97788-t78bl"
Jul 06 18:18:41.860635 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:41.860618    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/128b0f11-7a7d-44eb-9830-5208fb456e64-registry-tls\") pod \"image-registry-757fd97788-t78bl\" (UID: \"128b0f11-7a7d-44eb-9830-5208fb456e64\") " pod="openshift-image-registry/image-registry-757fd97788-t78bl"
Jul 06 18:18:41.860693 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:41.860663    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/128b0f11-7a7d-44eb-9830-5208fb456e64-image-registry-private-configuration\") pod \"image-registry-757fd97788-t78bl\" (UID: \"128b0f11-7a7d-44eb-9830-5208fb456e64\") " pod="openshift-image-registry/image-registry-757fd97788-t78bl"
Jul 06 18:18:41.866195 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:41.866170    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/128b0f11-7a7d-44eb-9830-5208fb456e64-bound-sa-token\") pod \"image-registry-757fd97788-t78bl\" (UID: \"128b0f11-7a7d-44eb-9830-5208fb456e64\") " pod="openshift-image-registry/image-registry-757fd97788-t78bl"
Jul 06 18:18:41.866377 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:41.866359    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r4r5s\" (UniqueName: \"kubernetes.io/projected/128b0f11-7a7d-44eb-9830-5208fb456e64-kube-api-access-r4r5s\") pod \"image-registry-757fd97788-t78bl\" (UID: \"128b0f11-7a7d-44eb-9830-5208fb456e64\") " pod="openshift-image-registry/image-registry-757fd97788-t78bl"
Jul 06 18:18:41.905109 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:41.905078    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7d5956586-ktr6l"
Jul 06 18:18:41.912841 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:41.912818    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5f4668c794-qxs4w"
Jul 06 18:18:41.917633 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:41.917605    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-744848fcbc-zc4nm"
Jul 06 18:18:42.021009 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:42.020858    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-757fd97788-t78bl"
Jul 06 18:18:42.045512 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:42.045124    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7d5956586-ktr6l"]
Jul 06 18:18:42.048708 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:18:42.048675    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddfe77255_7abf_49b6_acc4_0846089ad14e.slice/crio-362bb8806598122c247f3e85444609dc8a553fa778231dc7d71a9a88ec72ed4a WatchSource:0}: Error finding container 362bb8806598122c247f3e85444609dc8a553fa778231dc7d71a9a88ec72ed4a: Status 404 returned error can't find the container with id 362bb8806598122c247f3e85444609dc8a553fa778231dc7d71a9a88ec72ed4a
Jul 06 18:18:42.159832 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:42.157996    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-757fd97788-t78bl"]
Jul 06 18:18:42.161482 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:18:42.161459    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod128b0f11_7a7d_44eb_9830_5208fb456e64.slice/crio-76c991da012b86f59d4c4e33ee836c921e2e9262f3c681275d5b1ad3fb1f0ce6 WatchSource:0}: Error finding container 76c991da012b86f59d4c4e33ee836c921e2e9262f3c681275d5b1ad3fb1f0ce6: Status 404 returned error can't find the container with id 76c991da012b86f59d4c4e33ee836c921e2e9262f3c681275d5b1ad3fb1f0ce6
Jul 06 18:18:42.268054 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:42.268004    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-5f4668c794-qxs4w"]
Jul 06 18:18:42.270646 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:42.270617    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-744848fcbc-zc4nm"]
Jul 06 18:18:42.271240 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:18:42.271126    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod672e2368_ae59_4ab7_be3f_439daa363e9d.slice/crio-23896177903d059e2e1b811fcbaf803275553c6da1e63cf21d93e5e3c556372f WatchSource:0}: Error finding container 23896177903d059e2e1b811fcbaf803275553c6da1e63cf21d93e5e3c556372f: Status 404 returned error can't find the container with id 23896177903d059e2e1b811fcbaf803275553c6da1e63cf21d93e5e3c556372f
Jul 06 18:18:42.273557 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:18:42.273533    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddd383d1b_2e82_407d_84f5_09f902c288e1.slice/crio-7b7b181b7c7d929ad445f84ebdcbddde19f2b95be60bbd393aac0887bf06e0d9 WatchSource:0}: Error finding container 7b7b181b7c7d929ad445f84ebdcbddde19f2b95be60bbd393aac0887bf06e0d9: Status 404 returned error can't find the container with id 7b7b181b7c7d929ad445f84ebdcbddde19f2b95be60bbd393aac0887bf06e0d9
Jul 06 18:18:42.659557 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:42.659521    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-744848fcbc-zc4nm" event={"ID":"dd383d1b-2e82-407d-84f5-09f902c288e1","Type":"ContainerStarted","Data":"7b7b181b7c7d929ad445f84ebdcbddde19f2b95be60bbd393aac0887bf06e0d9"}
Jul 06 18:18:42.661045 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:42.661001    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-757fd97788-t78bl" event={"ID":"128b0f11-7a7d-44eb-9830-5208fb456e64","Type":"ContainerStarted","Data":"c76528dc7fa6f5e96eb090de23875b8d096ae66fd236d3349275ce6cc9464685"}
Jul 06 18:18:42.661166 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:42.661071    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-757fd97788-t78bl" event={"ID":"128b0f11-7a7d-44eb-9830-5208fb456e64","Type":"ContainerStarted","Data":"76c991da012b86f59d4c4e33ee836c921e2e9262f3c681275d5b1ad3fb1f0ce6"}
Jul 06 18:18:42.661166 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:42.661099    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-757fd97788-t78bl"
Jul 06 18:18:42.662132 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:42.662111    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5f4668c794-qxs4w" event={"ID":"672e2368-ae59-4ab7-be3f-439daa363e9d","Type":"ContainerStarted","Data":"23896177903d059e2e1b811fcbaf803275553c6da1e63cf21d93e5e3c556372f"}
Jul 06 18:18:42.663157 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:42.663136    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7d5956586-ktr6l" event={"ID":"dfe77255-7abf-49b6-acc4-0846089ad14e","Type":"ContainerStarted","Data":"362bb8806598122c247f3e85444609dc8a553fa778231dc7d71a9a88ec72ed4a"}
Jul 06 18:18:42.687075 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:42.687005    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-757fd97788-t78bl" podStartSLOduration=1.68698907 podStartE2EDuration="1.68698907s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:18:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:18:42.685513824 +0000 UTC m=+57.842912310" watchObservedRunningTime="2026-07-06 18:18:42.68698907 +0000 UTC m=+57.844387534"
Jul 06 18:18:44.575045 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:44.574641    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-9h479"
Jul 06 18:18:45.678576 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:45.678534    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7d5956586-ktr6l" event={"ID":"dfe77255-7abf-49b6-acc4-0846089ad14e","Type":"ContainerStarted","Data":"a757d630af8a9ddea59e65ad34c0b8329273caa2b7f98a329049262509935961"}
Jul 06 18:18:45.695615 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:45.695561    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7d5956586-ktr6l" podStartSLOduration=1.850999717 podStartE2EDuration="4.695545077s" totalImagesPullingTime="2.84454536s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:18:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:18:45.694695949 +0000 UTC m=+60.852094436" watchObservedRunningTime="2026-07-06 18:18:45.695545077 +0000 UTC m=+60.852943562"
Jul 06 18:18:47.685681 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:47.685641    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5f4668c794-qxs4w" event={"ID":"672e2368-ae59-4ab7-be3f-439daa363e9d","Type":"ContainerStarted","Data":"c4f904344427df2963e8bb067d53efc179d7c580748e785b6d32d8d37f62688e"}
Jul 06 18:18:47.686114 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:47.685906    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5f4668c794-qxs4w"
Jul 06 18:18:47.687484 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:47.687465    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5f4668c794-qxs4w"
Jul 06 18:18:47.702278 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:47.702235    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5f4668c794-qxs4w" podStartSLOduration=2.335727243 podStartE2EDuration="6.702223206s" totalImagesPullingTime="4.366495963s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:18:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:18:47.70073968 +0000 UTC m=+62.858138164" watchObservedRunningTime="2026-07-06 18:18:47.702223206 +0000 UTC m=+62.859621691"
Jul 06 18:18:48.651049 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:48.651001    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-lzzjn"
Jul 06 18:18:50.023277 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:50.023237    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-knzwx\" (UniqueName: \"kubernetes.io/projected/4e2f69c5-ef58-430a-8b74-eb53ce230c7e-kube-api-access-knzwx\") pod \"network-check-target-fp8fn\" (UID: \"4e2f69c5-ef58-430a-8b74-eb53ce230c7e\") " pod="openshift-network-diagnostics/network-check-target-fp8fn"
Jul 06 18:18:50.026236 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:50.026216    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 06 18:18:50.036475 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:50.036446    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 06 18:18:50.046302 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:50.046270    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-knzwx\" (UniqueName: \"kubernetes.io/projected/4e2f69c5-ef58-430a-8b74-eb53ce230c7e-kube-api-access-knzwx\") pod \"network-check-target-fp8fn\" (UID: \"4e2f69c5-ef58-430a-8b74-eb53ce230c7e\") " pod="openshift-network-diagnostics/network-check-target-fp8fn"
Jul 06 18:18:50.124473 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:50.124430    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3b6b323e-eca7-4c68-87a7-0d48a9114a2c-original-pull-secret\") pod \"global-pull-secret-syncer-xzqch\" (UID: \"3b6b323e-eca7-4c68-87a7-0d48a9114a2c\") " pod="kube-system/global-pull-secret-syncer-xzqch"
Jul 06 18:18:50.124473 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:50.124475    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f22387dc-2c30-4f80-8a80-3f6420c1c858-metrics-certs\") pod \"network-metrics-daemon-xjbzr\" (UID: \"f22387dc-2c30-4f80-8a80-3f6420c1c858\") " pod="openshift-multus/network-metrics-daemon-xjbzr"
Jul 06 18:18:50.127811 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:50.127785    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 06 18:18:50.127898 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:50.127880    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 06 18:18:50.132909 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:50.132895    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-m8lgm\""
Jul 06 18:18:50.137285 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:50.137264    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f22387dc-2c30-4f80-8a80-3f6420c1c858-metrics-certs\") pod \"network-metrics-daemon-xjbzr\" (UID: \"f22387dc-2c30-4f80-8a80-3f6420c1c858\") " pod="openshift-multus/network-metrics-daemon-xjbzr"
Jul 06 18:18:50.137364 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:50.137296    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3b6b323e-eca7-4c68-87a7-0d48a9114a2c-original-pull-secret\") pod \"global-pull-secret-syncer-xzqch\" (UID: \"3b6b323e-eca7-4c68-87a7-0d48a9114a2c\") " pod="kube-system/global-pull-secret-syncer-xzqch"
Jul 06 18:18:50.140223 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:50.140207    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fp8fn"
Jul 06 18:18:50.140300 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:50.140249    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xzqch"
Jul 06 18:18:50.279709 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:50.279635    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-fp8fn"]
Jul 06 18:18:50.284131 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:18:50.284100    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e2f69c5_ef58_430a_8b74_eb53ce230c7e.slice/crio-9eb6711ca8f1cb6173435425db0a42d29502d08b8c7525149f4d62b4735ad666 WatchSource:0}: Error finding container 9eb6711ca8f1cb6173435425db0a42d29502d08b8c7525149f4d62b4735ad666: Status 404 returned error can't find the container with id 9eb6711ca8f1cb6173435425db0a42d29502d08b8c7525149f4d62b4735ad666
Jul 06 18:18:50.295706 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:50.295684    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-xzqch"]
Jul 06 18:18:50.301432 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:18:50.301407    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b6b323e_eca7_4c68_87a7_0d48a9114a2c.slice/crio-92e6e1ed1563b614b44e65793fd3dc62ec94be0b1445e298d5e0664108c6b370 WatchSource:0}: Error finding container 92e6e1ed1563b614b44e65793fd3dc62ec94be0b1445e298d5e0664108c6b370: Status 404 returned error can't find the container with id 92e6e1ed1563b614b44e65793fd3dc62ec94be0b1445e298d5e0664108c6b370
Jul 06 18:18:50.437990 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:50.437957    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-4lbvz\""
Jul 06 18:18:50.445237 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:50.445219    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xjbzr"
Jul 06 18:18:50.568812 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:50.568741    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-xjbzr"]
Jul 06 18:18:50.571343 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:18:50.571319    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf22387dc_2c30_4f80_8a80_3f6420c1c858.slice/crio-65f71a2b018d53fa57563895c803cacb501cb8a3e2669d99c8aefcda8f60fbef WatchSource:0}: Error finding container 65f71a2b018d53fa57563895c803cacb501cb8a3e2669d99c8aefcda8f60fbef: Status 404 returned error can't find the container with id 65f71a2b018d53fa57563895c803cacb501cb8a3e2669d99c8aefcda8f60fbef
Jul 06 18:18:50.695756 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:50.695719    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-xzqch" event={"ID":"3b6b323e-eca7-4c68-87a7-0d48a9114a2c","Type":"ContainerStarted","Data":"92e6e1ed1563b614b44e65793fd3dc62ec94be0b1445e298d5e0664108c6b370"}
Jul 06 18:18:50.696771 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:50.696741    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-xjbzr" event={"ID":"f22387dc-2c30-4f80-8a80-3f6420c1c858","Type":"ContainerStarted","Data":"65f71a2b018d53fa57563895c803cacb501cb8a3e2669d99c8aefcda8f60fbef"}
Jul 06 18:18:50.697779 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:50.697752    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-fp8fn" event={"ID":"4e2f69c5-ef58-430a-8b74-eb53ce230c7e","Type":"ContainerStarted","Data":"9eb6711ca8f1cb6173435425db0a42d29502d08b8c7525149f4d62b4735ad666"}
Jul 06 18:18:51.654881 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:51.654847    2580 patch_prober.go:28] interesting pod/image-registry-58c4895898-bt6wr container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 06 18:18:51.655390 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:51.654907    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-58c4895898-bt6wr" podUID="cc40fe47-16bf-4a41-90e6-29f7e89f687f" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 06 18:18:52.144688 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:52.144176    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f354e957-d479-41ba-bf43-24303d500049-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-89h8f\" (UID: \"f354e957-d479-41ba-bf43-24303d500049\") " pod="openshift-insights/insights-runtime-extractor-89h8f"
Jul 06 18:18:52.147399 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:52.147373    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f354e957-d479-41ba-bf43-24303d500049-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-89h8f\" (UID: \"f354e957-d479-41ba-bf43-24303d500049\") " pod="openshift-insights/insights-runtime-extractor-89h8f"
Jul 06 18:18:52.358290 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:52.357994    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-6mvj2\""
Jul 06 18:18:52.364562 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:52.364526    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-89h8f"
Jul 06 18:18:52.528835 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:52.528727    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-89h8f"]
Jul 06 18:18:52.532710 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:18:52.532684    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf354e957_d479_41ba_bf43_24303d500049.slice/crio-90458cbb4e768f31178aaaed97acc9ae13f26b92db1cd4d3082865a612c2eebd WatchSource:0}: Error finding container 90458cbb4e768f31178aaaed97acc9ae13f26b92db1cd4d3082865a612c2eebd: Status 404 returned error can't find the container with id 90458cbb4e768f31178aaaed97acc9ae13f26b92db1cd4d3082865a612c2eebd
Jul 06 18:18:52.707549 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:52.707461    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-xjbzr" event={"ID":"f22387dc-2c30-4f80-8a80-3f6420c1c858","Type":"ContainerStarted","Data":"db6dbed100d8dcb37c05323287d2656f94e75cf7ca97fa56ca2e467acd7136ee"}
Jul 06 18:18:52.707549 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:52.707524    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-xjbzr" event={"ID":"f22387dc-2c30-4f80-8a80-3f6420c1c858","Type":"ContainerStarted","Data":"7f356130185e41d01cd77853820e96cb62000f77818a4305499523a27220cbf7"}
Jul 06 18:18:52.708772 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:52.708745    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-89h8f" event={"ID":"f354e957-d479-41ba-bf43-24303d500049","Type":"ContainerStarted","Data":"90458cbb4e768f31178aaaed97acc9ae13f26b92db1cd4d3082865a612c2eebd"}
Jul 06 18:18:55.529637 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:55.529577    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-xjbzr" podStartSLOduration=68.996461884 podStartE2EDuration="1m10.529552679s" totalImagesPullingTime="1.533090795s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:17:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:18:52.726217196 +0000 UTC m=+67.883615684" watchObservedRunningTime="2026-07-06 18:18:55.529552679 +0000 UTC m=+70.686951167"
Jul 06 18:18:55.530221 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:55.529872    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-r5gqj"]
Jul 06 18:18:55.559976 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:55.559947    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-r5gqj"
Jul 06 18:18:55.563213 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:55.563190    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 06 18:18:55.563213 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:55.563207    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 06 18:18:55.563433 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:55.563353    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 06 18:18:55.563433 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:55.563366    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 06 18:18:55.563433 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:55.563354    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 06 18:18:55.564619 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:55.564441    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-445cz\""
Jul 06 18:18:55.564619 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:55.564515    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 06 18:18:55.676155 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:55.676076    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/aed7c886-98ef-45f3-8b70-23a67b244fec-node-exporter-wtmp\") pod \"node-exporter-r5gqj\" (UID: \"aed7c886-98ef-45f3-8b70-23a67b244fec\") " pod="openshift-monitoring/node-exporter-r5gqj"
Jul 06 18:18:55.676155 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:55.676142    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xgg6q\" (UniqueName: \"kubernetes.io/projected/aed7c886-98ef-45f3-8b70-23a67b244fec-kube-api-access-xgg6q\") pod \"node-exporter-r5gqj\" (UID: \"aed7c886-98ef-45f3-8b70-23a67b244fec\") " pod="openshift-monitoring/node-exporter-r5gqj"
Jul 06 18:18:55.676623 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:55.676173    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/aed7c886-98ef-45f3-8b70-23a67b244fec-node-exporter-accelerators-collector-config\") pod \"node-exporter-r5gqj\" (UID: \"aed7c886-98ef-45f3-8b70-23a67b244fec\") " pod="openshift-monitoring/node-exporter-r5gqj"
Jul 06 18:18:55.676623 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:55.676211    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/aed7c886-98ef-45f3-8b70-23a67b244fec-node-exporter-textfile\") pod \"node-exporter-r5gqj\" (UID: \"aed7c886-98ef-45f3-8b70-23a67b244fec\") " pod="openshift-monitoring/node-exporter-r5gqj"
Jul 06 18:18:55.676623 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:55.676287    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/aed7c886-98ef-45f3-8b70-23a67b244fec-root\") pod \"node-exporter-r5gqj\" (UID: \"aed7c886-98ef-45f3-8b70-23a67b244fec\") " pod="openshift-monitoring/node-exporter-r5gqj"
Jul 06 18:18:55.676623 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:55.676316    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/aed7c886-98ef-45f3-8b70-23a67b244fec-metrics-client-ca\") pod \"node-exporter-r5gqj\" (UID: \"aed7c886-98ef-45f3-8b70-23a67b244fec\") " pod="openshift-monitoring/node-exporter-r5gqj"
Jul 06 18:18:55.676623 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:55.676403    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/aed7c886-98ef-45f3-8b70-23a67b244fec-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-r5gqj\" (UID: \"aed7c886-98ef-45f3-8b70-23a67b244fec\") " pod="openshift-monitoring/node-exporter-r5gqj"
Jul 06 18:18:55.676623 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:55.676430    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/aed7c886-98ef-45f3-8b70-23a67b244fec-sys\") pod \"node-exporter-r5gqj\" (UID: \"aed7c886-98ef-45f3-8b70-23a67b244fec\") " pod="openshift-monitoring/node-exporter-r5gqj"
Jul 06 18:18:55.676623 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:55.676501    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/aed7c886-98ef-45f3-8b70-23a67b244fec-node-exporter-tls\") pod \"node-exporter-r5gqj\" (UID: \"aed7c886-98ef-45f3-8b70-23a67b244fec\") " pod="openshift-monitoring/node-exporter-r5gqj"
Jul 06 18:18:55.777327 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:55.777295    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/aed7c886-98ef-45f3-8b70-23a67b244fec-root\") pod \"node-exporter-r5gqj\" (UID: \"aed7c886-98ef-45f3-8b70-23a67b244fec\") " pod="openshift-monitoring/node-exporter-r5gqj"
Jul 06 18:18:55.777327 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:55.777330    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/aed7c886-98ef-45f3-8b70-23a67b244fec-metrics-client-ca\") pod \"node-exporter-r5gqj\" (UID: \"aed7c886-98ef-45f3-8b70-23a67b244fec\") " pod="openshift-monitoring/node-exporter-r5gqj"
Jul 06 18:18:55.777569 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:55.777355    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/aed7c886-98ef-45f3-8b70-23a67b244fec-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-r5gqj\" (UID: \"aed7c886-98ef-45f3-8b70-23a67b244fec\") " pod="openshift-monitoring/node-exporter-r5gqj"
Jul 06 18:18:55.777569 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:55.777370    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/aed7c886-98ef-45f3-8b70-23a67b244fec-sys\") pod \"node-exporter-r5gqj\" (UID: \"aed7c886-98ef-45f3-8b70-23a67b244fec\") " pod="openshift-monitoring/node-exporter-r5gqj"
Jul 06 18:18:55.777569 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:55.777430    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/aed7c886-98ef-45f3-8b70-23a67b244fec-sys\") pod \"node-exporter-r5gqj\" (UID: \"aed7c886-98ef-45f3-8b70-23a67b244fec\") " pod="openshift-monitoring/node-exporter-r5gqj"
Jul 06 18:18:55.777569 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:55.777434    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/aed7c886-98ef-45f3-8b70-23a67b244fec-root\") pod \"node-exporter-r5gqj\" (UID: \"aed7c886-98ef-45f3-8b70-23a67b244fec\") " pod="openshift-monitoring/node-exporter-r5gqj"
Jul 06 18:18:55.777569 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:55.777485    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/aed7c886-98ef-45f3-8b70-23a67b244fec-node-exporter-tls\") pod \"node-exporter-r5gqj\" (UID: \"aed7c886-98ef-45f3-8b70-23a67b244fec\") " pod="openshift-monitoring/node-exporter-r5gqj"
Jul 06 18:18:55.777569 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:55.777532    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/aed7c886-98ef-45f3-8b70-23a67b244fec-node-exporter-wtmp\") pod \"node-exporter-r5gqj\" (UID: \"aed7c886-98ef-45f3-8b70-23a67b244fec\") " pod="openshift-monitoring/node-exporter-r5gqj"
Jul 06 18:18:55.777841 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:55.777578    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xgg6q\" (UniqueName: \"kubernetes.io/projected/aed7c886-98ef-45f3-8b70-23a67b244fec-kube-api-access-xgg6q\") pod \"node-exporter-r5gqj\" (UID: \"aed7c886-98ef-45f3-8b70-23a67b244fec\") " pod="openshift-monitoring/node-exporter-r5gqj"
Jul 06 18:18:55.777841 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:55.777606    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/aed7c886-98ef-45f3-8b70-23a67b244fec-node-exporter-accelerators-collector-config\") pod \"node-exporter-r5gqj\" (UID: \"aed7c886-98ef-45f3-8b70-23a67b244fec\") " pod="openshift-monitoring/node-exporter-r5gqj"
Jul 06 18:18:55.777841 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:55.777634    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/aed7c886-98ef-45f3-8b70-23a67b244fec-node-exporter-textfile\") pod \"node-exporter-r5gqj\" (UID: \"aed7c886-98ef-45f3-8b70-23a67b244fec\") " pod="openshift-monitoring/node-exporter-r5gqj"
Jul 06 18:18:55.777841 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:18:55.777656    2580 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 06 18:18:55.777841 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:18:55.777720    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aed7c886-98ef-45f3-8b70-23a67b244fec-node-exporter-tls podName:aed7c886-98ef-45f3-8b70-23a67b244fec nodeName:}" failed. No retries permitted until 2026-07-06 18:18:56.277697212 +0000 UTC m=+71.435095679 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/aed7c886-98ef-45f3-8b70-23a67b244fec-node-exporter-tls") pod "node-exporter-r5gqj" (UID: "aed7c886-98ef-45f3-8b70-23a67b244fec") : secret "node-exporter-tls" not found
Jul 06 18:18:55.778117 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:55.777927    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/aed7c886-98ef-45f3-8b70-23a67b244fec-metrics-client-ca\") pod \"node-exporter-r5gqj\" (UID: \"aed7c886-98ef-45f3-8b70-23a67b244fec\") " pod="openshift-monitoring/node-exporter-r5gqj"
Jul 06 18:18:55.778117 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:55.777927    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/aed7c886-98ef-45f3-8b70-23a67b244fec-node-exporter-textfile\") pod \"node-exporter-r5gqj\" (UID: \"aed7c886-98ef-45f3-8b70-23a67b244fec\") " pod="openshift-monitoring/node-exporter-r5gqj"
Jul 06 18:18:55.778117 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:55.778077    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/aed7c886-98ef-45f3-8b70-23a67b244fec-node-exporter-wtmp\") pod \"node-exporter-r5gqj\" (UID: \"aed7c886-98ef-45f3-8b70-23a67b244fec\") " pod="openshift-monitoring/node-exporter-r5gqj"
Jul 06 18:18:55.778314 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:55.778295    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/aed7c886-98ef-45f3-8b70-23a67b244fec-node-exporter-accelerators-collector-config\") pod \"node-exporter-r5gqj\" (UID: \"aed7c886-98ef-45f3-8b70-23a67b244fec\") " pod="openshift-monitoring/node-exporter-r5gqj"
Jul 06 18:18:55.780076 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:55.780005    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/aed7c886-98ef-45f3-8b70-23a67b244fec-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-r5gqj\" (UID: \"aed7c886-98ef-45f3-8b70-23a67b244fec\") " pod="openshift-monitoring/node-exporter-r5gqj"
Jul 06 18:18:55.787804 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:55.787782    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xgg6q\" (UniqueName: \"kubernetes.io/projected/aed7c886-98ef-45f3-8b70-23a67b244fec-kube-api-access-xgg6q\") pod \"node-exporter-r5gqj\" (UID: \"aed7c886-98ef-45f3-8b70-23a67b244fec\") " pod="openshift-monitoring/node-exporter-r5gqj"
Jul 06 18:18:56.285098 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:56.285052    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/aed7c886-98ef-45f3-8b70-23a67b244fec-node-exporter-tls\") pod \"node-exporter-r5gqj\" (UID: \"aed7c886-98ef-45f3-8b70-23a67b244fec\") " pod="openshift-monitoring/node-exporter-r5gqj"
Jul 06 18:18:56.287889 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:56.287860    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/aed7c886-98ef-45f3-8b70-23a67b244fec-node-exporter-tls\") pod \"node-exporter-r5gqj\" (UID: \"aed7c886-98ef-45f3-8b70-23a67b244fec\") " pod="openshift-monitoring/node-exporter-r5gqj"
Jul 06 18:18:56.473963 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:56.473747    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-r5gqj"
Jul 06 18:18:58.775685 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:18:58.775655    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaed7c886_98ef_45f3_8b70_23a67b244fec.slice/crio-460787b1a5dcefb0230aa60c2d61acddaa6755450ad71caf85889a910ee48542 WatchSource:0}: Error finding container 460787b1a5dcefb0230aa60c2d61acddaa6755450ad71caf85889a910ee48542: Status 404 returned error can't find the container with id 460787b1a5dcefb0230aa60c2d61acddaa6755450ad71caf85889a910ee48542
Jul 06 18:18:59.738551 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:59.738507    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-fp8fn" event={"ID":"4e2f69c5-ef58-430a-8b74-eb53ce230c7e","Type":"ContainerStarted","Data":"54c91d217e039e0ceec321885719d7be8723e642e43fc540830e4656019bd2c8"}
Jul 06 18:18:59.738733 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:59.738588    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-fp8fn"
Jul 06 18:18:59.740348 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:59.740310    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-89h8f" event={"ID":"f354e957-d479-41ba-bf43-24303d500049","Type":"ContainerStarted","Data":"5ad3658622cba386129573e3ce984020ac24e08ee4d3500812b3a5864150fb0e"}
Jul 06 18:18:59.742304 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:59.742242    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-r5gqj" event={"ID":"aed7c886-98ef-45f3-8b70-23a67b244fec","Type":"ContainerStarted","Data":"460787b1a5dcefb0230aa60c2d61acddaa6755450ad71caf85889a910ee48542"}
Jul 06 18:18:59.743838 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:59.743815    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-xzqch" event={"ID":"3b6b323e-eca7-4c68-87a7-0d48a9114a2c","Type":"ContainerStarted","Data":"7a9105cc3221e679fc94e72d300006de916158cf4dbe384c72a53fa634580831"}
Jul 06 18:18:59.758758 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:59.758699    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-fp8fn" podStartSLOduration=66.273398598 podStartE2EDuration="1m14.758679669s" totalImagesPullingTime="8.485281071s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:17:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:18:59.758006533 +0000 UTC m=+74.915405020" watchObservedRunningTime="2026-07-06 18:18:59.758679669 +0000 UTC m=+74.916078156"
Jul 06 18:18:59.778976 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:18:59.778920    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-xzqch" podStartSLOduration=66.309508729 podStartE2EDuration="1m14.77890231s" totalImagesPullingTime="8.469393581s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 18:17:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:18:59.777809294 +0000 UTC m=+74.935207785" watchObservedRunningTime="2026-07-06 18:18:59.77890231 +0000 UTC m=+74.936300795"
Jul 06 18:19:00.751062 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:19:00.751008    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-89h8f" event={"ID":"f354e957-d479-41ba-bf43-24303d500049","Type":"ContainerStarted","Data":"068caacb06f9d17b3ed039f84dfe619a33adc1a410053bebc6d64ab6e57c24b7"}
Jul 06 18:19:00.756182 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:19:00.756140    2580 generic.go:356] "Generic (PLEG): container finished" podID="aed7c886-98ef-45f3-8b70-23a67b244fec" containerID="575338df8d02b10ada89daa8db9d0283b7f493704f0edd825289fa30bb5185e1" exitCode=0
Jul 06 18:19:00.757936 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:19:00.757572    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-r5gqj" event={"ID":"aed7c886-98ef-45f3-8b70-23a67b244fec","Type":"ContainerDied","Data":"575338df8d02b10ada89daa8db9d0283b7f493704f0edd825289fa30bb5185e1"}
Jul 06 18:19:01.653713 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:19:01.653682    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-58c4895898-bt6wr"
Jul 06 18:19:01.762293 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:19:01.762248    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-r5gqj" event={"ID":"aed7c886-98ef-45f3-8b70-23a67b244fec","Type":"ContainerStarted","Data":"a971932ea6a54826965da60baf3bc9ff673e982a33116d96724854fc1ae15307"}
Jul 06 18:19:01.762483 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:19:01.762316    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-r5gqj" event={"ID":"aed7c886-98ef-45f3-8b70-23a67b244fec","Type":"ContainerStarted","Data":"94f90ca71ffd1abc45154c31d784a9e3465ca9c64dcf6ed1032f5888f1efd880"}
Jul 06 18:19:01.796364 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:19:01.796298    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-r5gqj" podStartSLOduration=5.597329566 podStartE2EDuration="6.796274288s" totalImagesPullingTime="1.12838557s" totalInitContainerRuntime="70.559152ms" isStatefulPod=true podCreationTimestamp="2026-07-06 18:18:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:19:01.795196624 +0000 UTC m=+76.952595111" watchObservedRunningTime="2026-07-06 18:19:01.796274288 +0000 UTC m=+76.953672775"
Jul 06 18:19:02.768345 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:19:02.768253    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-89h8f" event={"ID":"f354e957-d479-41ba-bf43-24303d500049","Type":"ContainerStarted","Data":"0e5cbedac39fd1dab413c26951a2835069d9f832afb199bfbc7abe2c8a151bbf"}
Jul 06 18:19:02.792795 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:19:02.792738    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-89h8f" podStartSLOduration=32.946367264 podStartE2EDuration="42.792719132s" totalImagesPullingTime="9.846351868s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 18:18:20 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:19:02.791683203 +0000 UTC m=+77.949081702" watchObservedRunningTime="2026-07-06 18:19:02.792719132 +0000 UTC m=+77.950117617"
Jul 06 18:19:03.674212 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:19:03.674181    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-757fd97788-t78bl"
Jul 06 18:19:06.666780 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:19:06.666733    2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-58c4895898-bt6wr" podUID="cc40fe47-16bf-4a41-90e6-29f7e89f687f" containerName="registry" containerID="cri-o://c91a73467725d2da41229740c9e90c20a4024e07c60f6862d54f7389fee5f844" gracePeriod=30
Jul 06 18:19:06.786283 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:19:06.786252    2580 generic.go:356] "Generic (PLEG): container finished" podID="cc40fe47-16bf-4a41-90e6-29f7e89f687f" containerID="c91a73467725d2da41229740c9e90c20a4024e07c60f6862d54f7389fee5f844" exitCode=0
Jul 06 18:19:06.786422 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:19:06.786334    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-58c4895898-bt6wr" event={"ID":"cc40fe47-16bf-4a41-90e6-29f7e89f687f","Type":"ContainerDied","Data":"c91a73467725d2da41229740c9e90c20a4024e07c60f6862d54f7389fee5f844"}
Jul 06 18:19:06.919221 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:19:06.919167    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-58c4895898-bt6wr"
Jul 06 18:19:07.079168 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:19:07.079126    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cc40fe47-16bf-4a41-90e6-29f7e89f687f-trusted-ca\") pod \"cc40fe47-16bf-4a41-90e6-29f7e89f687f\" (UID: \"cc40fe47-16bf-4a41-90e6-29f7e89f687f\") "
Jul 06 18:19:07.079356 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:19:07.079187    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n78h6\" (UniqueName: \"kubernetes.io/projected/cc40fe47-16bf-4a41-90e6-29f7e89f687f-kube-api-access-n78h6\") pod \"cc40fe47-16bf-4a41-90e6-29f7e89f687f\" (UID: \"cc40fe47-16bf-4a41-90e6-29f7e89f687f\") "
Jul 06 18:19:07.079356 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:19:07.079206    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cc40fe47-16bf-4a41-90e6-29f7e89f687f-bound-sa-token\") pod \"cc40fe47-16bf-4a41-90e6-29f7e89f687f\" (UID: \"cc40fe47-16bf-4a41-90e6-29f7e89f687f\") "
Jul 06 18:19:07.079356 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:19:07.079232    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/cc40fe47-16bf-4a41-90e6-29f7e89f687f-installation-pull-secrets\") pod \"cc40fe47-16bf-4a41-90e6-29f7e89f687f\" (UID: \"cc40fe47-16bf-4a41-90e6-29f7e89f687f\") "
Jul 06 18:19:07.079511 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:19:07.079410    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/cc40fe47-16bf-4a41-90e6-29f7e89f687f-registry-certificates\") pod \"cc40fe47-16bf-4a41-90e6-29f7e89f687f\" (UID: \"cc40fe47-16bf-4a41-90e6-29f7e89f687f\") "
Jul 06 18:19:07.079511 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:19:07.079499    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/cc40fe47-16bf-4a41-90e6-29f7e89f687f-image-registry-private-configuration\") pod \"cc40fe47-16bf-4a41-90e6-29f7e89f687f\" (UID: \"cc40fe47-16bf-4a41-90e6-29f7e89f687f\") "
Jul 06 18:19:07.079603 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:19:07.079527    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cc40fe47-16bf-4a41-90e6-29f7e89f687f-registry-tls\") pod \"cc40fe47-16bf-4a41-90e6-29f7e89f687f\" (UID: \"cc40fe47-16bf-4a41-90e6-29f7e89f687f\") "
Jul 06 18:19:07.079603 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:19:07.079555    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/cc40fe47-16bf-4a41-90e6-29f7e89f687f-ca-trust-extracted\") pod \"cc40fe47-16bf-4a41-90e6-29f7e89f687f\" (UID: \"cc40fe47-16bf-4a41-90e6-29f7e89f687f\") "
Jul 06 18:19:07.079850 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:19:07.079633    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cc40fe47-16bf-4a41-90e6-29f7e89f687f-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "cc40fe47-16bf-4a41-90e6-29f7e89f687f" (UID: "cc40fe47-16bf-4a41-90e6-29f7e89f687f"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 18:19:07.079850 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:19:07.079819    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cc40fe47-16bf-4a41-90e6-29f7e89f687f-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "cc40fe47-16bf-4a41-90e6-29f7e89f687f" (UID: "cc40fe47-16bf-4a41-90e6-29f7e89f687f"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 18:19:07.081976 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:19:07.081943    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cc40fe47-16bf-4a41-90e6-29f7e89f687f-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "cc40fe47-16bf-4a41-90e6-29f7e89f687f" (UID: "cc40fe47-16bf-4a41-90e6-29f7e89f687f"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 18:19:07.082104 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:19:07.082047    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cc40fe47-16bf-4a41-90e6-29f7e89f687f-kube-api-access-n78h6" (OuterVolumeSpecName: "kube-api-access-n78h6") pod "cc40fe47-16bf-4a41-90e6-29f7e89f687f" (UID: "cc40fe47-16bf-4a41-90e6-29f7e89f687f"). InnerVolumeSpecName "kube-api-access-n78h6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 18:19:07.082168 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:19:07.082095    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cc40fe47-16bf-4a41-90e6-29f7e89f687f-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "cc40fe47-16bf-4a41-90e6-29f7e89f687f" (UID: "cc40fe47-16bf-4a41-90e6-29f7e89f687f"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 18:19:07.082209 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:19:07.082169    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cc40fe47-16bf-4a41-90e6-29f7e89f687f-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "cc40fe47-16bf-4a41-90e6-29f7e89f687f" (UID: "cc40fe47-16bf-4a41-90e6-29f7e89f687f"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 18:19:07.082209 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:19:07.082195    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cc40fe47-16bf-4a41-90e6-29f7e89f687f-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "cc40fe47-16bf-4a41-90e6-29f7e89f687f" (UID: "cc40fe47-16bf-4a41-90e6-29f7e89f687f"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 18:19:07.088247 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:19:07.088224    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cc40fe47-16bf-4a41-90e6-29f7e89f687f-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "cc40fe47-16bf-4a41-90e6-29f7e89f687f" (UID: "cc40fe47-16bf-4a41-90e6-29f7e89f687f"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 18:19:07.180284 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:19:07.180206    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n78h6\" (UniqueName: \"kubernetes.io/projected/cc40fe47-16bf-4a41-90e6-29f7e89f687f-kube-api-access-n78h6\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:19:07.180284 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:19:07.180234    2580 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cc40fe47-16bf-4a41-90e6-29f7e89f687f-bound-sa-token\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:19:07.180284 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:19:07.180244    2580 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/cc40fe47-16bf-4a41-90e6-29f7e89f687f-installation-pull-secrets\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:19:07.180284 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:19:07.180254    2580 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/cc40fe47-16bf-4a41-90e6-29f7e89f687f-registry-certificates\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:19:07.180284 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:19:07.180264    2580 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/cc40fe47-16bf-4a41-90e6-29f7e89f687f-image-registry-private-configuration\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:19:07.180284 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:19:07.180273    2580 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cc40fe47-16bf-4a41-90e6-29f7e89f687f-registry-tls\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:19:07.180284 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:19:07.180281    2580 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/cc40fe47-16bf-4a41-90e6-29f7e89f687f-ca-trust-extracted\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:19:07.180284 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:19:07.180290    2580 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cc40fe47-16bf-4a41-90e6-29f7e89f687f-trusted-ca\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:19:07.790856 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:19:07.790820    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-58c4895898-bt6wr" event={"ID":"cc40fe47-16bf-4a41-90e6-29f7e89f687f","Type":"ContainerDied","Data":"81848c3c59e98869f705fe9b7f27e18a6fc13a602f85ce41b5503c468f37faac"}
Jul 06 18:19:07.791333 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:19:07.790886    2580 scope.go:117] "RemoveContainer" containerID="c91a73467725d2da41229740c9e90c20a4024e07c60f6862d54f7389fee5f844"
Jul 06 18:19:07.791333 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:19:07.790840    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-58c4895898-bt6wr"
Jul 06 18:19:07.810160 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:19:07.810128    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-58c4895898-bt6wr"]
Jul 06 18:19:07.815881 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:19:07.815844    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-58c4895898-bt6wr"]
Jul 06 18:19:09.321235 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:19:09.321189    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cc40fe47-16bf-4a41-90e6-29f7e89f687f" path="/var/lib/kubelet/pods/cc40fe47-16bf-4a41-90e6-29f7e89f687f/volumes"
Jul 06 18:19:11.807909 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:19:11.807866    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-744848fcbc-zc4nm" event={"ID":"dd383d1b-2e82-407d-84f5-09f902c288e1","Type":"ContainerStarted","Data":"aba6942ce0662b2ebe2695cd94859ca93195b6e03e1b5a2cd4589a12440203f7"}
Jul 06 18:19:11.808310 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:19:11.807921    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-744848fcbc-zc4nm" event={"ID":"dd383d1b-2e82-407d-84f5-09f902c288e1","Type":"ContainerStarted","Data":"0f63e4d198cff9657062f750307c734ff0d38bc085b2d27079f197e0916b93eb"}
Jul 06 18:19:11.808310 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:19:11.807944    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-744848fcbc-zc4nm" event={"ID":"dd383d1b-2e82-407d-84f5-09f902c288e1","Type":"ContainerStarted","Data":"c1ea7ca248e1869ca427a96abb7965b0665e3608fe1b18959ded6562f494143d"}
Jul 06 18:19:11.851176 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:19:11.851123    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-744848fcbc-zc4nm" podStartSLOduration=2.18504863 podStartE2EDuration="30.851105507s" totalImagesPullingTime="28.666056877s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:18:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:19:11.850499951 +0000 UTC m=+87.007898438" watchObservedRunningTime="2026-07-06 18:19:11.851105507 +0000 UTC m=+87.008503992"
Jul 06 18:19:30.759638 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:19:30.759604    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-fp8fn"
Jul 06 18:19:37.585245 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:19:37.585214    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-r5gqj_aed7c886-98ef-45f3-8b70-23a67b244fec/init-textfile/0.log"
Jul 06 18:19:37.785705 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:19:37.785667    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-r5gqj_aed7c886-98ef-45f3-8b70-23a67b244fec/node-exporter/0.log"
Jul 06 18:19:37.984415 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:19:37.984336    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-r5gqj_aed7c886-98ef-45f3-8b70-23a67b244fec/kube-rbac-proxy/0.log"
Jul 06 18:22:45.210170 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:22:45.210141    2580 kubelet.go:1648] "Image garbage collection succeeded"
Jul 06 18:23:51.269851 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:23:51.269820    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6bfdf69c66-tbm2m"]
Jul 06 18:23:51.270293 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:23:51.270063    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cc40fe47-16bf-4a41-90e6-29f7e89f687f" containerName="registry"
Jul 06 18:23:51.270293 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:23:51.270074    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="cc40fe47-16bf-4a41-90e6-29f7e89f687f" containerName="registry"
Jul 06 18:23:51.270293 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:23:51.270117    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="cc40fe47-16bf-4a41-90e6-29f7e89f687f" containerName="registry"
Jul 06 18:23:51.272699 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:23:51.272682    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6bfdf69c66-tbm2m"
Jul 06 18:23:51.276440 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:23:51.276410    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 06 18:23:51.277420 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:23:51.277402    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-mhwpf\""
Jul 06 18:23:51.277672 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:23:51.277651    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 06 18:23:51.277743 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:23:51.277660    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 06 18:23:51.277803 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:23:51.277768    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 06 18:23:51.306934 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:23:51.306895    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6bfdf69c66-tbm2m"]
Jul 06 18:23:51.367635 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:23:51.367605    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2p288\" (UniqueName: \"kubernetes.io/projected/6bc2ec3e-8ccd-41a1-a9d8-6013a0535dfb-kube-api-access-2p288\") pod \"opendatahub-operator-controller-manager-6bfdf69c66-tbm2m\" (UID: \"6bc2ec3e-8ccd-41a1-a9d8-6013a0535dfb\") " pod="opendatahub/opendatahub-operator-controller-manager-6bfdf69c66-tbm2m"
Jul 06 18:23:51.367784 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:23:51.367639    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6bc2ec3e-8ccd-41a1-a9d8-6013a0535dfb-webhook-cert\") pod \"opendatahub-operator-controller-manager-6bfdf69c66-tbm2m\" (UID: \"6bc2ec3e-8ccd-41a1-a9d8-6013a0535dfb\") " pod="opendatahub/opendatahub-operator-controller-manager-6bfdf69c66-tbm2m"
Jul 06 18:23:51.367784 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:23:51.367672    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6bc2ec3e-8ccd-41a1-a9d8-6013a0535dfb-tmp\") pod \"opendatahub-operator-controller-manager-6bfdf69c66-tbm2m\" (UID: \"6bc2ec3e-8ccd-41a1-a9d8-6013a0535dfb\") " pod="opendatahub/opendatahub-operator-controller-manager-6bfdf69c66-tbm2m"
Jul 06 18:23:51.367784 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:23:51.367767    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/6bc2ec3e-8ccd-41a1-a9d8-6013a0535dfb-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6bfdf69c66-tbm2m\" (UID: \"6bc2ec3e-8ccd-41a1-a9d8-6013a0535dfb\") " pod="opendatahub/opendatahub-operator-controller-manager-6bfdf69c66-tbm2m"
Jul 06 18:23:51.367889 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:23:51.367803    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/6bc2ec3e-8ccd-41a1-a9d8-6013a0535dfb-manifests\") pod \"opendatahub-operator-controller-manager-6bfdf69c66-tbm2m\" (UID: \"6bc2ec3e-8ccd-41a1-a9d8-6013a0535dfb\") " pod="opendatahub/opendatahub-operator-controller-manager-6bfdf69c66-tbm2m"
Jul 06 18:23:51.468414 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:23:51.468379    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/6bc2ec3e-8ccd-41a1-a9d8-6013a0535dfb-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6bfdf69c66-tbm2m\" (UID: \"6bc2ec3e-8ccd-41a1-a9d8-6013a0535dfb\") " pod="opendatahub/opendatahub-operator-controller-manager-6bfdf69c66-tbm2m"
Jul 06 18:23:51.468601 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:23:51.468427    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/6bc2ec3e-8ccd-41a1-a9d8-6013a0535dfb-manifests\") pod \"opendatahub-operator-controller-manager-6bfdf69c66-tbm2m\" (UID: \"6bc2ec3e-8ccd-41a1-a9d8-6013a0535dfb\") " pod="opendatahub/opendatahub-operator-controller-manager-6bfdf69c66-tbm2m"
Jul 06 18:23:51.468601 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:23:51.468451    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2p288\" (UniqueName: \"kubernetes.io/projected/6bc2ec3e-8ccd-41a1-a9d8-6013a0535dfb-kube-api-access-2p288\") pod \"opendatahub-operator-controller-manager-6bfdf69c66-tbm2m\" (UID: \"6bc2ec3e-8ccd-41a1-a9d8-6013a0535dfb\") " pod="opendatahub/opendatahub-operator-controller-manager-6bfdf69c66-tbm2m"
Jul 06 18:23:51.468601 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:23:51.468467    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6bc2ec3e-8ccd-41a1-a9d8-6013a0535dfb-webhook-cert\") pod \"opendatahub-operator-controller-manager-6bfdf69c66-tbm2m\" (UID: \"6bc2ec3e-8ccd-41a1-a9d8-6013a0535dfb\") " pod="opendatahub/opendatahub-operator-controller-manager-6bfdf69c66-tbm2m"
Jul 06 18:23:51.468601 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:23:51.468489    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6bc2ec3e-8ccd-41a1-a9d8-6013a0535dfb-tmp\") pod \"opendatahub-operator-controller-manager-6bfdf69c66-tbm2m\" (UID: \"6bc2ec3e-8ccd-41a1-a9d8-6013a0535dfb\") " pod="opendatahub/opendatahub-operator-controller-manager-6bfdf69c66-tbm2m"
Jul 06 18:23:51.468885 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:23:51.468860    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/6bc2ec3e-8ccd-41a1-a9d8-6013a0535dfb-manifests\") pod \"opendatahub-operator-controller-manager-6bfdf69c66-tbm2m\" (UID: \"6bc2ec3e-8ccd-41a1-a9d8-6013a0535dfb\") " pod="opendatahub/opendatahub-operator-controller-manager-6bfdf69c66-tbm2m"
Jul 06 18:23:51.470968 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:23:51.470945    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6bc2ec3e-8ccd-41a1-a9d8-6013a0535dfb-tmp\") pod \"opendatahub-operator-controller-manager-6bfdf69c66-tbm2m\" (UID: \"6bc2ec3e-8ccd-41a1-a9d8-6013a0535dfb\") " pod="opendatahub/opendatahub-operator-controller-manager-6bfdf69c66-tbm2m"
Jul 06 18:23:51.471309 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:23:51.471294    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6bc2ec3e-8ccd-41a1-a9d8-6013a0535dfb-webhook-cert\") pod \"opendatahub-operator-controller-manager-6bfdf69c66-tbm2m\" (UID: \"6bc2ec3e-8ccd-41a1-a9d8-6013a0535dfb\") " pod="opendatahub/opendatahub-operator-controller-manager-6bfdf69c66-tbm2m"
Jul 06 18:23:51.471471 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:23:51.471449    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/6bc2ec3e-8ccd-41a1-a9d8-6013a0535dfb-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6bfdf69c66-tbm2m\" (UID: \"6bc2ec3e-8ccd-41a1-a9d8-6013a0535dfb\") " pod="opendatahub/opendatahub-operator-controller-manager-6bfdf69c66-tbm2m"
Jul 06 18:23:51.481612 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:23:51.481584    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2p288\" (UniqueName: \"kubernetes.io/projected/6bc2ec3e-8ccd-41a1-a9d8-6013a0535dfb-kube-api-access-2p288\") pod \"opendatahub-operator-controller-manager-6bfdf69c66-tbm2m\" (UID: \"6bc2ec3e-8ccd-41a1-a9d8-6013a0535dfb\") " pod="opendatahub/opendatahub-operator-controller-manager-6bfdf69c66-tbm2m"
Jul 06 18:23:51.582225 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:23:51.582189    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6bfdf69c66-tbm2m"
Jul 06 18:23:51.714823 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:23:51.714761    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6bfdf69c66-tbm2m"]
Jul 06 18:23:51.717598 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:23:51.717572    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6bc2ec3e_8ccd_41a1_a9d8_6013a0535dfb.slice/crio-f15332b6521dfa14f7ae73d06943c2c9a433a26f366605dae03202c89790aac2 WatchSource:0}: Error finding container f15332b6521dfa14f7ae73d06943c2c9a433a26f366605dae03202c89790aac2: Status 404 returned error can't find the container with id f15332b6521dfa14f7ae73d06943c2c9a433a26f366605dae03202c89790aac2
Jul 06 18:23:51.719338 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:23:51.719322    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 06 18:23:52.710298 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:23:52.710264    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6bfdf69c66-tbm2m" event={"ID":"6bc2ec3e-8ccd-41a1-a9d8-6013a0535dfb","Type":"ContainerStarted","Data":"f15332b6521dfa14f7ae73d06943c2c9a433a26f366605dae03202c89790aac2"}
Jul 06 18:23:57.729285 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:23:57.729243    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6bfdf69c66-tbm2m" event={"ID":"6bc2ec3e-8ccd-41a1-a9d8-6013a0535dfb","Type":"ContainerStarted","Data":"0152631498bb98a28d22cd983a66c96db0a4e24fc48404d4bcdc6de407e80413"}
Jul 06 18:23:58.733987 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:23:58.733949    2580 generic.go:356] "Generic (PLEG): container finished" podID="6bc2ec3e-8ccd-41a1-a9d8-6013a0535dfb" containerID="0152631498bb98a28d22cd983a66c96db0a4e24fc48404d4bcdc6de407e80413" exitCode=0
Jul 06 18:23:58.734378 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:23:58.734038    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6bfdf69c66-tbm2m" event={"ID":"6bc2ec3e-8ccd-41a1-a9d8-6013a0535dfb","Type":"ContainerDied","Data":"0152631498bb98a28d22cd983a66c96db0a4e24fc48404d4bcdc6de407e80413"}
Jul 06 18:23:59.739215 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:23:59.739177    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6bfdf69c66-tbm2m" event={"ID":"6bc2ec3e-8ccd-41a1-a9d8-6013a0535dfb","Type":"ContainerStarted","Data":"3b98aabf02ebf78f57261a93ef51178b56525b64eb140bddb926090c1884a2dd"}
Jul 06 18:23:59.739672 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:23:59.739252    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-6bfdf69c66-tbm2m"
Jul 06 18:23:59.762303 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:23:59.762257    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-6bfdf69c66-tbm2m" podStartSLOduration=2.035269035 podStartE2EDuration="8.762241116s" totalImagesPullingTime="5.631570106s" totalInitContainerRuntime="1.095401975s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:23:51 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:23:59.75982816 +0000 UTC m=+374.917226646" watchObservedRunningTime="2026-07-06 18:23:59.762241116 +0000 UTC m=+374.919639601"
Jul 06 18:24:10.744593 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:10.744562    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-6bfdf69c66-tbm2m"
Jul 06 18:24:12.971656 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:12.971609    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-68bf6db468-pvqn9"]
Jul 06 18:24:13.013385 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:13.013348    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-68bf6db468-pvqn9"]
Jul 06 18:24:13.013545 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:13.013442    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-68bf6db468-pvqn9"
Jul 06 18:24:13.019349 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:13.019313    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6f86a0d6-2cde-4e56-8348-9ad3351f9500-tls-certs\") pod \"kube-auth-proxy-68bf6db468-pvqn9\" (UID: \"6f86a0d6-2cde-4e56-8348-9ad3351f9500\") " pod="openshift-ingress/kube-auth-proxy-68bf6db468-pvqn9"
Jul 06 18:24:13.019487 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:13.019457    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wzls9\" (UniqueName: \"kubernetes.io/projected/6f86a0d6-2cde-4e56-8348-9ad3351f9500-kube-api-access-wzls9\") pod \"kube-auth-proxy-68bf6db468-pvqn9\" (UID: \"6f86a0d6-2cde-4e56-8348-9ad3351f9500\") " pod="openshift-ingress/kube-auth-proxy-68bf6db468-pvqn9"
Jul 06 18:24:13.019607 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:13.019497    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6f86a0d6-2cde-4e56-8348-9ad3351f9500-tmp\") pod \"kube-auth-proxy-68bf6db468-pvqn9\" (UID: \"6f86a0d6-2cde-4e56-8348-9ad3351f9500\") " pod="openshift-ingress/kube-auth-proxy-68bf6db468-pvqn9"
Jul 06 18:24:13.020537 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:13.020507    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 06 18:24:13.020537 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:13.020527    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 06 18:24:13.020694 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:13.020577    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-6lwvb\""
Jul 06 18:24:13.020694 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:13.020600    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 06 18:24:13.028081 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:13.028063    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 06 18:24:13.120431 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:13.120385    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wzls9\" (UniqueName: \"kubernetes.io/projected/6f86a0d6-2cde-4e56-8348-9ad3351f9500-kube-api-access-wzls9\") pod \"kube-auth-proxy-68bf6db468-pvqn9\" (UID: \"6f86a0d6-2cde-4e56-8348-9ad3351f9500\") " pod="openshift-ingress/kube-auth-proxy-68bf6db468-pvqn9"
Jul 06 18:24:13.120431 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:13.120433    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6f86a0d6-2cde-4e56-8348-9ad3351f9500-tmp\") pod \"kube-auth-proxy-68bf6db468-pvqn9\" (UID: \"6f86a0d6-2cde-4e56-8348-9ad3351f9500\") " pod="openshift-ingress/kube-auth-proxy-68bf6db468-pvqn9"
Jul 06 18:24:13.120636 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:13.120472    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6f86a0d6-2cde-4e56-8348-9ad3351f9500-tls-certs\") pod \"kube-auth-proxy-68bf6db468-pvqn9\" (UID: \"6f86a0d6-2cde-4e56-8348-9ad3351f9500\") " pod="openshift-ingress/kube-auth-proxy-68bf6db468-pvqn9"
Jul 06 18:24:13.122696 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:13.122675    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6f86a0d6-2cde-4e56-8348-9ad3351f9500-tmp\") pod \"kube-auth-proxy-68bf6db468-pvqn9\" (UID: \"6f86a0d6-2cde-4e56-8348-9ad3351f9500\") " pod="openshift-ingress/kube-auth-proxy-68bf6db468-pvqn9"
Jul 06 18:24:13.122988 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:13.122969    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6f86a0d6-2cde-4e56-8348-9ad3351f9500-tls-certs\") pod \"kube-auth-proxy-68bf6db468-pvqn9\" (UID: \"6f86a0d6-2cde-4e56-8348-9ad3351f9500\") " pod="openshift-ingress/kube-auth-proxy-68bf6db468-pvqn9"
Jul 06 18:24:13.132182 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:13.132158    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wzls9\" (UniqueName: \"kubernetes.io/projected/6f86a0d6-2cde-4e56-8348-9ad3351f9500-kube-api-access-wzls9\") pod \"kube-auth-proxy-68bf6db468-pvqn9\" (UID: \"6f86a0d6-2cde-4e56-8348-9ad3351f9500\") " pod="openshift-ingress/kube-auth-proxy-68bf6db468-pvqn9"
Jul 06 18:24:13.292806 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:13.292709    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-cjcm8"]
Jul 06 18:24:13.323479 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:13.323446    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-68bf6db468-pvqn9"
Jul 06 18:24:13.329907 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:13.329875    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-cjcm8"]
Jul 06 18:24:13.330058 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:13.330028    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-cjcm8"
Jul 06 18:24:13.332623 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:13.332605    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Jul 06 18:24:13.332754 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:13.332716    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-7j6mf\""
Jul 06 18:24:13.423555 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:13.423522    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b77b21f6-57a3-476e-96c0-f9c87a61683e-cert\") pod \"odh-model-controller-7998c67dc4-cjcm8\" (UID: \"b77b21f6-57a3-476e-96c0-f9c87a61683e\") " pod="opendatahub/odh-model-controller-7998c67dc4-cjcm8"
Jul 06 18:24:13.423709 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:13.423595    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sgdrj\" (UniqueName: \"kubernetes.io/projected/b77b21f6-57a3-476e-96c0-f9c87a61683e-kube-api-access-sgdrj\") pod \"odh-model-controller-7998c67dc4-cjcm8\" (UID: \"b77b21f6-57a3-476e-96c0-f9c87a61683e\") " pod="opendatahub/odh-model-controller-7998c67dc4-cjcm8"
Jul 06 18:24:13.463097 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:13.463065    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-68bf6db468-pvqn9"]
Jul 06 18:24:13.466306 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:24:13.466272    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f86a0d6_2cde_4e56_8348_9ad3351f9500.slice/crio-5390d401a9f2ec5e87c853f6440b09ffaec5309bc2acb59263067e7eeb6e7406 WatchSource:0}: Error finding container 5390d401a9f2ec5e87c853f6440b09ffaec5309bc2acb59263067e7eeb6e7406: Status 404 returned error can't find the container with id 5390d401a9f2ec5e87c853f6440b09ffaec5309bc2acb59263067e7eeb6e7406
Jul 06 18:24:13.524891 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:13.524861    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sgdrj\" (UniqueName: \"kubernetes.io/projected/b77b21f6-57a3-476e-96c0-f9c87a61683e-kube-api-access-sgdrj\") pod \"odh-model-controller-7998c67dc4-cjcm8\" (UID: \"b77b21f6-57a3-476e-96c0-f9c87a61683e\") " pod="opendatahub/odh-model-controller-7998c67dc4-cjcm8"
Jul 06 18:24:13.525091 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:13.524909    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b77b21f6-57a3-476e-96c0-f9c87a61683e-cert\") pod \"odh-model-controller-7998c67dc4-cjcm8\" (UID: \"b77b21f6-57a3-476e-96c0-f9c87a61683e\") " pod="opendatahub/odh-model-controller-7998c67dc4-cjcm8"
Jul 06 18:24:13.525091 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:24:13.525003    2580 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Jul 06 18:24:13.525091 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:24:13.525081    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b77b21f6-57a3-476e-96c0-f9c87a61683e-cert podName:b77b21f6-57a3-476e-96c0-f9c87a61683e nodeName:}" failed. No retries permitted until 2026-07-06 18:24:14.02505684 +0000 UTC m=+389.182455304 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/b77b21f6-57a3-476e-96c0-f9c87a61683e-cert") pod "odh-model-controller-7998c67dc4-cjcm8" (UID: "b77b21f6-57a3-476e-96c0-f9c87a61683e") : secret "odh-model-controller-webhook-cert" not found
Jul 06 18:24:13.534380 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:13.534358    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sgdrj\" (UniqueName: \"kubernetes.io/projected/b77b21f6-57a3-476e-96c0-f9c87a61683e-kube-api-access-sgdrj\") pod \"odh-model-controller-7998c67dc4-cjcm8\" (UID: \"b77b21f6-57a3-476e-96c0-f9c87a61683e\") " pod="opendatahub/odh-model-controller-7998c67dc4-cjcm8"
Jul 06 18:24:13.786333 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:13.786294    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-68bf6db468-pvqn9" event={"ID":"6f86a0d6-2cde-4e56-8348-9ad3351f9500","Type":"ContainerStarted","Data":"5390d401a9f2ec5e87c853f6440b09ffaec5309bc2acb59263067e7eeb6e7406"}
Jul 06 18:24:14.028030 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:14.027973    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b77b21f6-57a3-476e-96c0-f9c87a61683e-cert\") pod \"odh-model-controller-7998c67dc4-cjcm8\" (UID: \"b77b21f6-57a3-476e-96c0-f9c87a61683e\") " pod="opendatahub/odh-model-controller-7998c67dc4-cjcm8"
Jul 06 18:24:14.030503 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:14.030477    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b77b21f6-57a3-476e-96c0-f9c87a61683e-cert\") pod \"odh-model-controller-7998c67dc4-cjcm8\" (UID: \"b77b21f6-57a3-476e-96c0-f9c87a61683e\") " pod="opendatahub/odh-model-controller-7998c67dc4-cjcm8"
Jul 06 18:24:14.197009 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:14.196970    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-rt527"]
Jul 06 18:24:14.199349 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:14.199327    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-rt527"
Jul 06 18:24:14.202910 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:14.202886    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Jul 06 18:24:14.215874 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:14.215174    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-rt527"]
Jul 06 18:24:14.229966 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:14.229934    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6wxkr\" (UniqueName: \"kubernetes.io/projected/a116c1ff-d114-4ba8-8a6d-eec4317c6978-kube-api-access-6wxkr\") pod \"model-serving-api-54d89bd5fc-rt527\" (UID: \"a116c1ff-d114-4ba8-8a6d-eec4317c6978\") " pod="opendatahub/model-serving-api-54d89bd5fc-rt527"
Jul 06 18:24:14.230135 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:14.230044    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a116c1ff-d114-4ba8-8a6d-eec4317c6978-tls-certs\") pod \"model-serving-api-54d89bd5fc-rt527\" (UID: \"a116c1ff-d114-4ba8-8a6d-eec4317c6978\") " pod="opendatahub/model-serving-api-54d89bd5fc-rt527"
Jul 06 18:24:14.241824 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:14.241796    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-cjcm8"
Jul 06 18:24:14.331282 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:14.331249    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6wxkr\" (UniqueName: \"kubernetes.io/projected/a116c1ff-d114-4ba8-8a6d-eec4317c6978-kube-api-access-6wxkr\") pod \"model-serving-api-54d89bd5fc-rt527\" (UID: \"a116c1ff-d114-4ba8-8a6d-eec4317c6978\") " pod="opendatahub/model-serving-api-54d89bd5fc-rt527"
Jul 06 18:24:14.331453 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:14.331335    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a116c1ff-d114-4ba8-8a6d-eec4317c6978-tls-certs\") pod \"model-serving-api-54d89bd5fc-rt527\" (UID: \"a116c1ff-d114-4ba8-8a6d-eec4317c6978\") " pod="opendatahub/model-serving-api-54d89bd5fc-rt527"
Jul 06 18:24:14.331453 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:24:14.331434    2580 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found
Jul 06 18:24:14.331561 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:24:14.331498    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a116c1ff-d114-4ba8-8a6d-eec4317c6978-tls-certs podName:a116c1ff-d114-4ba8-8a6d-eec4317c6978 nodeName:}" failed. No retries permitted until 2026-07-06 18:24:14.831477399 +0000 UTC m=+389.988875884 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/a116c1ff-d114-4ba8-8a6d-eec4317c6978-tls-certs") pod "model-serving-api-54d89bd5fc-rt527" (UID: "a116c1ff-d114-4ba8-8a6d-eec4317c6978") : secret "model-serving-api-tls" not found
Jul 06 18:24:14.346000 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:14.345967    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-64rch"]
Jul 06 18:24:14.348280 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:14.348254    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-64rch"
Jul 06 18:24:14.349287 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:14.349264    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6wxkr\" (UniqueName: \"kubernetes.io/projected/a116c1ff-d114-4ba8-8a6d-eec4317c6978-kube-api-access-6wxkr\") pod \"model-serving-api-54d89bd5fc-rt527\" (UID: \"a116c1ff-d114-4ba8-8a6d-eec4317c6978\") " pod="opendatahub/model-serving-api-54d89bd5fc-rt527"
Jul 06 18:24:14.351952 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:14.351666    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 06 18:24:14.351952 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:14.351687    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 06 18:24:14.351952 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:14.351688    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 06 18:24:14.351952 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:14.351790    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-227ks\""
Jul 06 18:24:14.351952 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:14.351675    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 06 18:24:14.363044 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:14.362658    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-64rch"]
Jul 06 18:24:14.394283 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:14.394071    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-cjcm8"]
Jul 06 18:24:14.397206 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:24:14.397154    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb77b21f6_57a3_476e_96c0_f9c87a61683e.slice/crio-c8f9355db55d9bbfe502783aa0e8a19aaa698fdc3ab2485ecc1f1db885a2bfba WatchSource:0}: Error finding container c8f9355db55d9bbfe502783aa0e8a19aaa698fdc3ab2485ecc1f1db885a2bfba: Status 404 returned error can't find the container with id c8f9355db55d9bbfe502783aa0e8a19aaa698fdc3ab2485ecc1f1db885a2bfba
Jul 06 18:24:14.432664 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:14.432631    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/9ff4b8e0-05cb-4047-a327-a49b76896bcf-cacerts\") pod \"istiod-openshift-gateway-7574459656-64rch\" (UID: \"9ff4b8e0-05cb-4047-a327-a49b76896bcf\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-64rch"
Jul 06 18:24:14.432828 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:14.432680    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/9ff4b8e0-05cb-4047-a327-a49b76896bcf-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-64rch\" (UID: \"9ff4b8e0-05cb-4047-a327-a49b76896bcf\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-64rch"
Jul 06 18:24:14.432828 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:14.432750    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/9ff4b8e0-05cb-4047-a327-a49b76896bcf-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-64rch\" (UID: \"9ff4b8e0-05cb-4047-a327-a49b76896bcf\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-64rch"
Jul 06 18:24:14.432828 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:14.432796    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9ff4b8e0-05cb-4047-a327-a49b76896bcf-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-64rch\" (UID: \"9ff4b8e0-05cb-4047-a327-a49b76896bcf\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-64rch"
Jul 06 18:24:14.432980 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:14.432850    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lbdgn\" (UniqueName: \"kubernetes.io/projected/9ff4b8e0-05cb-4047-a327-a49b76896bcf-kube-api-access-lbdgn\") pod \"istiod-openshift-gateway-7574459656-64rch\" (UID: \"9ff4b8e0-05cb-4047-a327-a49b76896bcf\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-64rch"
Jul 06 18:24:14.432980 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:14.432902    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9ff4b8e0-05cb-4047-a327-a49b76896bcf-istio-token\") pod \"istiod-openshift-gateway-7574459656-64rch\" (UID: \"9ff4b8e0-05cb-4047-a327-a49b76896bcf\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-64rch"
Jul 06 18:24:14.432980 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:14.432928    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/9ff4b8e0-05cb-4047-a327-a49b76896bcf-local-certs\") pod \"istiod-openshift-gateway-7574459656-64rch\" (UID: \"9ff4b8e0-05cb-4047-a327-a49b76896bcf\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-64rch"
Jul 06 18:24:14.533917 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:14.533830    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lbdgn\" (UniqueName: \"kubernetes.io/projected/9ff4b8e0-05cb-4047-a327-a49b76896bcf-kube-api-access-lbdgn\") pod \"istiod-openshift-gateway-7574459656-64rch\" (UID: \"9ff4b8e0-05cb-4047-a327-a49b76896bcf\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-64rch"
Jul 06 18:24:14.533917 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:14.533888    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9ff4b8e0-05cb-4047-a327-a49b76896bcf-istio-token\") pod \"istiod-openshift-gateway-7574459656-64rch\" (UID: \"9ff4b8e0-05cb-4047-a327-a49b76896bcf\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-64rch"
Jul 06 18:24:14.533917 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:14.533913    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/9ff4b8e0-05cb-4047-a327-a49b76896bcf-local-certs\") pod \"istiod-openshift-gateway-7574459656-64rch\" (UID: \"9ff4b8e0-05cb-4047-a327-a49b76896bcf\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-64rch"
Jul 06 18:24:14.534204 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:14.533938    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/9ff4b8e0-05cb-4047-a327-a49b76896bcf-cacerts\") pod \"istiod-openshift-gateway-7574459656-64rch\" (UID: \"9ff4b8e0-05cb-4047-a327-a49b76896bcf\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-64rch"
Jul 06 18:24:14.534204 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:14.533968    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/9ff4b8e0-05cb-4047-a327-a49b76896bcf-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-64rch\" (UID: \"9ff4b8e0-05cb-4047-a327-a49b76896bcf\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-64rch"
Jul 06 18:24:14.534204 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:14.534172    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/9ff4b8e0-05cb-4047-a327-a49b76896bcf-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-64rch\" (UID: \"9ff4b8e0-05cb-4047-a327-a49b76896bcf\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-64rch"
Jul 06 18:24:14.534352 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:14.534234    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9ff4b8e0-05cb-4047-a327-a49b76896bcf-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-64rch\" (UID: \"9ff4b8e0-05cb-4047-a327-a49b76896bcf\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-64rch"
Jul 06 18:24:14.534697 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:14.534667    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/9ff4b8e0-05cb-4047-a327-a49b76896bcf-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-64rch\" (UID: \"9ff4b8e0-05cb-4047-a327-a49b76896bcf\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-64rch"
Jul 06 18:24:14.536237 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:14.536212    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/9ff4b8e0-05cb-4047-a327-a49b76896bcf-local-certs\") pod \"istiod-openshift-gateway-7574459656-64rch\" (UID: \"9ff4b8e0-05cb-4047-a327-a49b76896bcf\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-64rch"
Jul 06 18:24:14.536388 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:14.536368    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9ff4b8e0-05cb-4047-a327-a49b76896bcf-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-64rch\" (UID: \"9ff4b8e0-05cb-4047-a327-a49b76896bcf\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-64rch"
Jul 06 18:24:14.536443 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:14.536370    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/9ff4b8e0-05cb-4047-a327-a49b76896bcf-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-64rch\" (UID: \"9ff4b8e0-05cb-4047-a327-a49b76896bcf\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-64rch"
Jul 06 18:24:14.536443 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:14.536402    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/9ff4b8e0-05cb-4047-a327-a49b76896bcf-cacerts\") pod \"istiod-openshift-gateway-7574459656-64rch\" (UID: \"9ff4b8e0-05cb-4047-a327-a49b76896bcf\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-64rch"
Jul 06 18:24:14.541713 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:14.541692    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9ff4b8e0-05cb-4047-a327-a49b76896bcf-istio-token\") pod \"istiod-openshift-gateway-7574459656-64rch\" (UID: \"9ff4b8e0-05cb-4047-a327-a49b76896bcf\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-64rch"
Jul 06 18:24:14.541920 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:14.541901    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lbdgn\" (UniqueName: \"kubernetes.io/projected/9ff4b8e0-05cb-4047-a327-a49b76896bcf-kube-api-access-lbdgn\") pod \"istiod-openshift-gateway-7574459656-64rch\" (UID: \"9ff4b8e0-05cb-4047-a327-a49b76896bcf\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-64rch"
Jul 06 18:24:14.672647 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:14.672614    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-64rch"
Jul 06 18:24:14.792122 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:14.792030    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-cjcm8" event={"ID":"b77b21f6-57a3-476e-96c0-f9c87a61683e","Type":"ContainerStarted","Data":"c8f9355db55d9bbfe502783aa0e8a19aaa698fdc3ab2485ecc1f1db885a2bfba"}
Jul 06 18:24:14.815562 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:14.815526    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-64rch"]
Jul 06 18:24:14.820414 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:24:14.820391    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ff4b8e0_05cb_4047_a327_a49b76896bcf.slice/crio-11966cc99b630ed8a1314564e879eed909e969983f4949bc466bc37e22b05e04 WatchSource:0}: Error finding container 11966cc99b630ed8a1314564e879eed909e969983f4949bc466bc37e22b05e04: Status 404 returned error can't find the container with id 11966cc99b630ed8a1314564e879eed909e969983f4949bc466bc37e22b05e04
Jul 06 18:24:14.837474 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:14.837447    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a116c1ff-d114-4ba8-8a6d-eec4317c6978-tls-certs\") pod \"model-serving-api-54d89bd5fc-rt527\" (UID: \"a116c1ff-d114-4ba8-8a6d-eec4317c6978\") " pod="opendatahub/model-serving-api-54d89bd5fc-rt527"
Jul 06 18:24:14.839636 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:14.839619    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a116c1ff-d114-4ba8-8a6d-eec4317c6978-tls-certs\") pod \"model-serving-api-54d89bd5fc-rt527\" (UID: \"a116c1ff-d114-4ba8-8a6d-eec4317c6978\") " pod="opendatahub/model-serving-api-54d89bd5fc-rt527"
Jul 06 18:24:15.112844 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:15.112800    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-rt527"
Jul 06 18:24:15.278645 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:15.278611    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-rt527"]
Jul 06 18:24:15.281460 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:24:15.281303    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda116c1ff_d114_4ba8_8a6d_eec4317c6978.slice/crio-8b630b275ceafcee58251c6da99a0502f057be763ed2cc5e94cd01a975095459 WatchSource:0}: Error finding container 8b630b275ceafcee58251c6da99a0502f057be763ed2cc5e94cd01a975095459: Status 404 returned error can't find the container with id 8b630b275ceafcee58251c6da99a0502f057be763ed2cc5e94cd01a975095459
Jul 06 18:24:15.799628 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:15.799584    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-rt527" event={"ID":"a116c1ff-d114-4ba8-8a6d-eec4317c6978","Type":"ContainerStarted","Data":"8b630b275ceafcee58251c6da99a0502f057be763ed2cc5e94cd01a975095459"}
Jul 06 18:24:15.801528 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:15.801469    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-64rch" event={"ID":"9ff4b8e0-05cb-4047-a327-a49b76896bcf","Type":"ContainerStarted","Data":"11966cc99b630ed8a1314564e879eed909e969983f4949bc466bc37e22b05e04"}
Jul 06 18:24:19.488947 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:19.488897    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 06 18:24:19.489476 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:19.488983    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 06 18:24:19.820576 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:19.820475    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-cjcm8" event={"ID":"b77b21f6-57a3-476e-96c0-f9c87a61683e","Type":"ContainerStarted","Data":"74569c97fc8f60190bf9823b13c7df29b8213a5e14a8b45c8a78dba199a9bb72"}
Jul 06 18:24:19.820576 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:19.820585    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-cjcm8"
Jul 06 18:24:19.822230 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:19.822201    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-rt527" event={"ID":"a116c1ff-d114-4ba8-8a6d-eec4317c6978","Type":"ContainerStarted","Data":"188a0409842e37afd33d6ad206208f02407fdd229c9563c0c5f03e55ea984882"}
Jul 06 18:24:19.822376 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:19.822281    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-rt527"
Jul 06 18:24:19.823882 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:19.823855    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-64rch" event={"ID":"9ff4b8e0-05cb-4047-a327-a49b76896bcf","Type":"ContainerStarted","Data":"f800fb53b94784a4635655668dc6dfcd0eac68a211d24bc99e5d36219cd57790"}
Jul 06 18:24:19.824027 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:19.823905    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-64rch"
Jul 06 18:24:19.825485 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:19.825463    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-68bf6db468-pvqn9" event={"ID":"6f86a0d6-2cde-4e56-8348-9ad3351f9500","Type":"ContainerStarted","Data":"2a2c2b692438290d89d91c2c9803911eb53c0e3f13eb45cff1240543ddb8a8a3"}
Jul 06 18:24:19.842312 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:19.842258    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-cjcm8" podStartSLOduration=1.7535092190000001 podStartE2EDuration="6.842236579s" totalImagesPullingTime="5.08872736s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:24:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:24:19.840378215 +0000 UTC m=+394.997776700" watchObservedRunningTime="2026-07-06 18:24:19.842236579 +0000 UTC m=+394.999635066"
Jul 06 18:24:19.866563 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:19.866511    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-rt527" podStartSLOduration=1.671531903 podStartE2EDuration="5.866493933s" totalImagesPullingTime="4.19496203s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:24:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:24:19.866259246 +0000 UTC m=+395.023657731" watchObservedRunningTime="2026-07-06 18:24:19.866493933 +0000 UTC m=+395.023892418"
Jul 06 18:24:19.892687 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:19.892640    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-64rch" podStartSLOduration=1.226808409 podStartE2EDuration="5.892624516s" totalImagesPullingTime="4.665816107s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:24:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:24:19.891040101 +0000 UTC m=+395.048438582" watchObservedRunningTime="2026-07-06 18:24:19.892624516 +0000 UTC m=+395.050023001"
Jul 06 18:24:19.912955 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:19.912900    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-68bf6db468-pvqn9" podStartSLOduration=1.9025845540000001 podStartE2EDuration="7.912880551s" totalImagesPullingTime="6.010295997s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:24:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:24:19.911753096 +0000 UTC m=+395.069151584" watchObservedRunningTime="2026-07-06 18:24:19.912880551 +0000 UTC m=+395.070279042"
Jul 06 18:24:20.831342 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:20.831307    2580 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-64rch container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 06 18:24:20.831706 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:20.831358    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-64rch" podUID="9ff4b8e0-05cb-4047-a327-a49b76896bcf" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 06 18:24:21.178952 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:21.178843    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4m2pkz"]
Jul 06 18:24:21.182205 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:21.182181    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4m2pkz"
Jul 06 18:24:21.186077 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:21.186053    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-2ffkq\""
Jul 06 18:24:21.205237 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:21.205179    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4m2pkz"]
Jul 06 18:24:21.294623 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:21.294577    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zgqwr\" (UniqueName: \"kubernetes.io/projected/5d7cc344-17c6-4259-90cf-4422e436112b-kube-api-access-zgqwr\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4m2pkz\" (UID: \"5d7cc344-17c6-4259-90cf-4422e436112b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4m2pkz"
Jul 06 18:24:21.294623 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:21.294624    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5d7cc344-17c6-4259-90cf-4422e436112b-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4m2pkz\" (UID: \"5d7cc344-17c6-4259-90cf-4422e436112b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4m2pkz"
Jul 06 18:24:21.294845 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:21.294652    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5d7cc344-17c6-4259-90cf-4422e436112b-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4m2pkz\" (UID: \"5d7cc344-17c6-4259-90cf-4422e436112b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4m2pkz"
Jul 06 18:24:21.294845 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:21.294700    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5d7cc344-17c6-4259-90cf-4422e436112b-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4m2pkz\" (UID: \"5d7cc344-17c6-4259-90cf-4422e436112b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4m2pkz"
Jul 06 18:24:21.294845 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:21.294731    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d7cc344-17c6-4259-90cf-4422e436112b-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4m2pkz\" (UID: \"5d7cc344-17c6-4259-90cf-4422e436112b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4m2pkz"
Jul 06 18:24:21.294845 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:21.294770    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d7cc344-17c6-4259-90cf-4422e436112b-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4m2pkz\" (UID: \"5d7cc344-17c6-4259-90cf-4422e436112b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4m2pkz"
Jul 06 18:24:21.294845 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:21.294790    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5d7cc344-17c6-4259-90cf-4422e436112b-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4m2pkz\" (UID: \"5d7cc344-17c6-4259-90cf-4422e436112b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4m2pkz"
Jul 06 18:24:21.294845 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:21.294821    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5d7cc344-17c6-4259-90cf-4422e436112b-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4m2pkz\" (UID: \"5d7cc344-17c6-4259-90cf-4422e436112b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4m2pkz"
Jul 06 18:24:21.295056 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:21.294848    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5d7cc344-17c6-4259-90cf-4422e436112b-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4m2pkz\" (UID: \"5d7cc344-17c6-4259-90cf-4422e436112b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4m2pkz"
Jul 06 18:24:21.395794 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:21.395753    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5d7cc344-17c6-4259-90cf-4422e436112b-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4m2pkz\" (UID: \"5d7cc344-17c6-4259-90cf-4422e436112b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4m2pkz"
Jul 06 18:24:21.395968 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:21.395805    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zgqwr\" (UniqueName: \"kubernetes.io/projected/5d7cc344-17c6-4259-90cf-4422e436112b-kube-api-access-zgqwr\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4m2pkz\" (UID: \"5d7cc344-17c6-4259-90cf-4422e436112b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4m2pkz"
Jul 06 18:24:21.395968 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:21.395916    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5d7cc344-17c6-4259-90cf-4422e436112b-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4m2pkz\" (UID: \"5d7cc344-17c6-4259-90cf-4422e436112b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4m2pkz"
Jul 06 18:24:21.395968 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:21.395965    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5d7cc344-17c6-4259-90cf-4422e436112b-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4m2pkz\" (UID: \"5d7cc344-17c6-4259-90cf-4422e436112b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4m2pkz"
Jul 06 18:24:21.396158 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:21.395985    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5d7cc344-17c6-4259-90cf-4422e436112b-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4m2pkz\" (UID: \"5d7cc344-17c6-4259-90cf-4422e436112b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4m2pkz"
Jul 06 18:24:21.396158 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:21.396002    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d7cc344-17c6-4259-90cf-4422e436112b-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4m2pkz\" (UID: \"5d7cc344-17c6-4259-90cf-4422e436112b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4m2pkz"
Jul 06 18:24:21.396158 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:21.396060    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d7cc344-17c6-4259-90cf-4422e436112b-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4m2pkz\" (UID: \"5d7cc344-17c6-4259-90cf-4422e436112b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4m2pkz"
Jul 06 18:24:21.396158 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:21.396076    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5d7cc344-17c6-4259-90cf-4422e436112b-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4m2pkz\" (UID: \"5d7cc344-17c6-4259-90cf-4422e436112b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4m2pkz"
Jul 06 18:24:21.396158 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:21.396125    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5d7cc344-17c6-4259-90cf-4422e436112b-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4m2pkz\" (UID: \"5d7cc344-17c6-4259-90cf-4422e436112b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4m2pkz"
Jul 06 18:24:21.396455 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:21.396427    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5d7cc344-17c6-4259-90cf-4422e436112b-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4m2pkz\" (UID: \"5d7cc344-17c6-4259-90cf-4422e436112b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4m2pkz"
Jul 06 18:24:21.396570 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:21.396455    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d7cc344-17c6-4259-90cf-4422e436112b-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4m2pkz\" (UID: \"5d7cc344-17c6-4259-90cf-4422e436112b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4m2pkz"
Jul 06 18:24:21.396570 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:21.396526    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5d7cc344-17c6-4259-90cf-4422e436112b-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4m2pkz\" (UID: \"5d7cc344-17c6-4259-90cf-4422e436112b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4m2pkz"
Jul 06 18:24:21.396690 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:21.396616    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d7cc344-17c6-4259-90cf-4422e436112b-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4m2pkz\" (UID: \"5d7cc344-17c6-4259-90cf-4422e436112b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4m2pkz"
Jul 06 18:24:21.397053 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:21.397008    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5d7cc344-17c6-4259-90cf-4422e436112b-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4m2pkz\" (UID: \"5d7cc344-17c6-4259-90cf-4422e436112b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4m2pkz"
Jul 06 18:24:21.398679 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:21.398659    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5d7cc344-17c6-4259-90cf-4422e436112b-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4m2pkz\" (UID: \"5d7cc344-17c6-4259-90cf-4422e436112b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4m2pkz"
Jul 06 18:24:21.398775 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:21.398725    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5d7cc344-17c6-4259-90cf-4422e436112b-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4m2pkz\" (UID: \"5d7cc344-17c6-4259-90cf-4422e436112b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4m2pkz"
Jul 06 18:24:21.415681 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:21.415654    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5d7cc344-17c6-4259-90cf-4422e436112b-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4m2pkz\" (UID: \"5d7cc344-17c6-4259-90cf-4422e436112b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4m2pkz"
Jul 06 18:24:21.425430 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:21.425401    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zgqwr\" (UniqueName: \"kubernetes.io/projected/5d7cc344-17c6-4259-90cf-4422e436112b-kube-api-access-zgqwr\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4m2pkz\" (UID: \"5d7cc344-17c6-4259-90cf-4422e436112b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4m2pkz"
Jul 06 18:24:21.492759 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:21.492675    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4m2pkz"
Jul 06 18:24:21.636632 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:21.636596    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4m2pkz"]
Jul 06 18:24:21.639947 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:24:21.639917    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d7cc344_17c6_4259_90cf_4422e436112b.slice/crio-e6e09a2af642c97378516e2b29172abd5d7d9c57db67d07e8bc7851ac1c665ff WatchSource:0}: Error finding container e6e09a2af642c97378516e2b29172abd5d7d9c57db67d07e8bc7851ac1c665ff: Status 404 returned error can't find the container with id e6e09a2af642c97378516e2b29172abd5d7d9c57db67d07e8bc7851ac1c665ff
Jul 06 18:24:21.840653 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:21.840615    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4m2pkz" event={"ID":"5d7cc344-17c6-4259-90cf-4422e436112b","Type":"ContainerStarted","Data":"e6e09a2af642c97378516e2b29172abd5d7d9c57db67d07e8bc7851ac1c665ff"}
Jul 06 18:24:23.831412 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:23.831375    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-64rch"
Jul 06 18:24:24.144979 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:24.144891    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-c977q"]
Jul 06 18:24:24.152245 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:24.152215    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-c977q"
Jul 06 18:24:24.156436 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:24.156277    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-f8ddq\""
Jul 06 18:24:24.156605 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:24.156583    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Jul 06 18:24:24.173667 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:24.173637    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-c977q"]
Jul 06 18:24:24.213476 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:24.213439    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-4vm88"]
Jul 06 18:24:24.218682 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:24.218652    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-4vm88"]
Jul 06 18:24:24.218822 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:24.218792    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4vm88"
Jul 06 18:24:24.222777 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:24.222637    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fdcv7\" (UniqueName: \"kubernetes.io/projected/dfe46fe3-2f4d-4855-97c1-38e4e974c6f8-kube-api-access-fdcv7\") pod \"kserve-controller-manager-d6f4b65b8-c977q\" (UID: \"dfe46fe3-2f4d-4855-97c1-38e4e974c6f8\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-c977q"
Jul 06 18:24:24.222777 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:24.222706    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/dfe46fe3-2f4d-4855-97c1-38e4e974c6f8-cert\") pod \"kserve-controller-manager-d6f4b65b8-c977q\" (UID: \"dfe46fe3-2f4d-4855-97c1-38e4e974c6f8\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-c977q"
Jul 06 18:24:24.224530 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:24.224507    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-9fkck\""
Jul 06 18:24:24.224530 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:24.224516    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Jul 06 18:24:24.324220 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:24.324174    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fdcv7\" (UniqueName: \"kubernetes.io/projected/dfe46fe3-2f4d-4855-97c1-38e4e974c6f8-kube-api-access-fdcv7\") pod \"kserve-controller-manager-d6f4b65b8-c977q\" (UID: \"dfe46fe3-2f4d-4855-97c1-38e4e974c6f8\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-c977q"
Jul 06 18:24:24.324375 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:24.324244    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hl425\" (UniqueName: \"kubernetes.io/projected/e9cabd49-cdd7-4882-afc0-da156d82c239-kube-api-access-hl425\") pod \"llmisvc-controller-manager-777c6bd7d5-4vm88\" (UID: \"e9cabd49-cdd7-4882-afc0-da156d82c239\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4vm88"
Jul 06 18:24:24.324375 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:24.324285    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e9cabd49-cdd7-4882-afc0-da156d82c239-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-4vm88\" (UID: \"e9cabd49-cdd7-4882-afc0-da156d82c239\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4vm88"
Jul 06 18:24:24.324375 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:24.324319    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/dfe46fe3-2f4d-4855-97c1-38e4e974c6f8-cert\") pod \"kserve-controller-manager-d6f4b65b8-c977q\" (UID: \"dfe46fe3-2f4d-4855-97c1-38e4e974c6f8\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-c977q"
Jul 06 18:24:24.324512 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:24:24.324426    2580 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found
Jul 06 18:24:24.324512 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:24:24.324506    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dfe46fe3-2f4d-4855-97c1-38e4e974c6f8-cert podName:dfe46fe3-2f4d-4855-97c1-38e4e974c6f8 nodeName:}" failed. No retries permitted until 2026-07-06 18:24:24.824479764 +0000 UTC m=+399.981878242 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/dfe46fe3-2f4d-4855-97c1-38e4e974c6f8-cert") pod "kserve-controller-manager-d6f4b65b8-c977q" (UID: "dfe46fe3-2f4d-4855-97c1-38e4e974c6f8") : secret "kserve-webhook-server-cert" not found
Jul 06 18:24:24.335785 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:24.335754    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fdcv7\" (UniqueName: \"kubernetes.io/projected/dfe46fe3-2f4d-4855-97c1-38e4e974c6f8-kube-api-access-fdcv7\") pod \"kserve-controller-manager-d6f4b65b8-c977q\" (UID: \"dfe46fe3-2f4d-4855-97c1-38e4e974c6f8\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-c977q"
Jul 06 18:24:24.425362 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:24.425275    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hl425\" (UniqueName: \"kubernetes.io/projected/e9cabd49-cdd7-4882-afc0-da156d82c239-kube-api-access-hl425\") pod \"llmisvc-controller-manager-777c6bd7d5-4vm88\" (UID: \"e9cabd49-cdd7-4882-afc0-da156d82c239\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4vm88"
Jul 06 18:24:24.425362 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:24.425317    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e9cabd49-cdd7-4882-afc0-da156d82c239-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-4vm88\" (UID: \"e9cabd49-cdd7-4882-afc0-da156d82c239\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4vm88"
Jul 06 18:24:24.427648 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:24.427626    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e9cabd49-cdd7-4882-afc0-da156d82c239-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-4vm88\" (UID: \"e9cabd49-cdd7-4882-afc0-da156d82c239\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4vm88"
Jul 06 18:24:24.434673 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:24.434651    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hl425\" (UniqueName: \"kubernetes.io/projected/e9cabd49-cdd7-4882-afc0-da156d82c239-kube-api-access-hl425\") pod \"llmisvc-controller-manager-777c6bd7d5-4vm88\" (UID: \"e9cabd49-cdd7-4882-afc0-da156d82c239\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4vm88"
Jul 06 18:24:24.464753 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:24.464720    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 06 18:24:24.464860 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:24.464804    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 06 18:24:24.464860 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:24.464850    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 06 18:24:24.532072 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:24.532048    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4vm88"
Jul 06 18:24:24.696127 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:24.696101    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-4vm88"]
Jul 06 18:24:24.698150 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:24:24.698120    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pode9cabd49_cdd7_4882_afc0_da156d82c239.slice/crio-89a7534d38910a66ffefab5e99200aea8d7d6672baa973debe422642c4bf49c2 WatchSource:0}: Error finding container 89a7534d38910a66ffefab5e99200aea8d7d6672baa973debe422642c4bf49c2: Status 404 returned error can't find the container with id 89a7534d38910a66ffefab5e99200aea8d7d6672baa973debe422642c4bf49c2
Jul 06 18:24:24.829117 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:24.829059    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/dfe46fe3-2f4d-4855-97c1-38e4e974c6f8-cert\") pod \"kserve-controller-manager-d6f4b65b8-c977q\" (UID: \"dfe46fe3-2f4d-4855-97c1-38e4e974c6f8\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-c977q"
Jul 06 18:24:24.831389 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:24.831367    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/dfe46fe3-2f4d-4855-97c1-38e4e974c6f8-cert\") pod \"kserve-controller-manager-d6f4b65b8-c977q\" (UID: \"dfe46fe3-2f4d-4855-97c1-38e4e974c6f8\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-c977q"
Jul 06 18:24:24.854707 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:24.854672    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4m2pkz" event={"ID":"5d7cc344-17c6-4259-90cf-4422e436112b","Type":"ContainerStarted","Data":"b43ae4b03664ebddf20cf947b4f39b4e9af97ff95d59383915e2b082ccc611b1"}
Jul 06 18:24:24.855780 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:24.855760    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4vm88" event={"ID":"e9cabd49-cdd7-4882-afc0-da156d82c239","Type":"ContainerStarted","Data":"89a7534d38910a66ffefab5e99200aea8d7d6672baa973debe422642c4bf49c2"}
Jul 06 18:24:24.877394 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:24.877346    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4m2pkz" podStartSLOduration=1.054955415 podStartE2EDuration="3.877331515s" totalImagesPullingTime="2.8223761s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:24:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:24:24.876159367 +0000 UTC m=+400.033557853" watchObservedRunningTime="2026-07-06 18:24:24.877331515 +0000 UTC m=+400.034730000"
Jul 06 18:24:25.065770 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:25.065686    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-c977q"
Jul 06 18:24:25.207450 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:25.207425    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-c977q"]
Jul 06 18:24:25.209305 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:24:25.209279    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddfe46fe3_2f4d_4855_97c1_38e4e974c6f8.slice/crio-e7aef20c45f540092ac833a19f1fe1c55b5b64ec8ed7e3ddb95dcde51fc4f880 WatchSource:0}: Error finding container e7aef20c45f540092ac833a19f1fe1c55b5b64ec8ed7e3ddb95dcde51fc4f880: Status 404 returned error can't find the container with id e7aef20c45f540092ac833a19f1fe1c55b5b64ec8ed7e3ddb95dcde51fc4f880
Jul 06 18:24:25.493095 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:25.493058    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4m2pkz"
Jul 06 18:24:25.498096 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:25.498072    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4m2pkz"
Jul 06 18:24:25.863010 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:25.862956    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-c977q" event={"ID":"dfe46fe3-2f4d-4855-97c1-38e4e974c6f8","Type":"ContainerStarted","Data":"e7aef20c45f540092ac833a19f1fe1c55b5b64ec8ed7e3ddb95dcde51fc4f880"}
Jul 06 18:24:25.863479 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:25.863213    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4m2pkz"
Jul 06 18:24:25.864624 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:25.864600    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4m2pkz"
Jul 06 18:24:25.966707 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:25.966678    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-64rch"]
Jul 06 18:24:25.966918 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:25.966896    2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-64rch" podUID="9ff4b8e0-05cb-4047-a327-a49b76896bcf" containerName="discovery" containerID="cri-o://f800fb53b94784a4635655668dc6dfcd0eac68a211d24bc99e5d36219cd57790" gracePeriod=30
Jul 06 18:24:26.286426 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:26.286397    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-64rch"
Jul 06 18:24:26.341782 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:26.341744    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9ff4b8e0-05cb-4047-a327-a49b76896bcf-istio-kubeconfig\") pod \"9ff4b8e0-05cb-4047-a327-a49b76896bcf\" (UID: \"9ff4b8e0-05cb-4047-a327-a49b76896bcf\") "
Jul 06 18:24:26.341782 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:26.341781    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lbdgn\" (UniqueName: \"kubernetes.io/projected/9ff4b8e0-05cb-4047-a327-a49b76896bcf-kube-api-access-lbdgn\") pod \"9ff4b8e0-05cb-4047-a327-a49b76896bcf\" (UID: \"9ff4b8e0-05cb-4047-a327-a49b76896bcf\") "
Jul 06 18:24:26.342033 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:26.341813    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9ff4b8e0-05cb-4047-a327-a49b76896bcf-istio-token\") pod \"9ff4b8e0-05cb-4047-a327-a49b76896bcf\" (UID: \"9ff4b8e0-05cb-4047-a327-a49b76896bcf\") "
Jul 06 18:24:26.342033 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:26.341847    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/9ff4b8e0-05cb-4047-a327-a49b76896bcf-istio-csr-ca-configmap\") pod \"9ff4b8e0-05cb-4047-a327-a49b76896bcf\" (UID: \"9ff4b8e0-05cb-4047-a327-a49b76896bcf\") "
Jul 06 18:24:26.342033 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:26.341876    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/9ff4b8e0-05cb-4047-a327-a49b76896bcf-cacerts\") pod \"9ff4b8e0-05cb-4047-a327-a49b76896bcf\" (UID: \"9ff4b8e0-05cb-4047-a327-a49b76896bcf\") "
Jul 06 18:24:26.342033 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:26.341901    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/9ff4b8e0-05cb-4047-a327-a49b76896bcf-local-certs\") pod \"9ff4b8e0-05cb-4047-a327-a49b76896bcf\" (UID: \"9ff4b8e0-05cb-4047-a327-a49b76896bcf\") "
Jul 06 18:24:26.342033 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:26.341920    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/9ff4b8e0-05cb-4047-a327-a49b76896bcf-istio-csr-dns-cert\") pod \"9ff4b8e0-05cb-4047-a327-a49b76896bcf\" (UID: \"9ff4b8e0-05cb-4047-a327-a49b76896bcf\") "
Jul 06 18:24:26.342404 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:26.342353    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9ff4b8e0-05cb-4047-a327-a49b76896bcf-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "9ff4b8e0-05cb-4047-a327-a49b76896bcf" (UID: "9ff4b8e0-05cb-4047-a327-a49b76896bcf"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 18:24:26.344373 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:26.344333    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9ff4b8e0-05cb-4047-a327-a49b76896bcf-kube-api-access-lbdgn" (OuterVolumeSpecName: "kube-api-access-lbdgn") pod "9ff4b8e0-05cb-4047-a327-a49b76896bcf" (UID: "9ff4b8e0-05cb-4047-a327-a49b76896bcf"). InnerVolumeSpecName "kube-api-access-lbdgn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 18:24:26.344373 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:26.344354    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9ff4b8e0-05cb-4047-a327-a49b76896bcf-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "9ff4b8e0-05cb-4047-a327-a49b76896bcf" (UID: "9ff4b8e0-05cb-4047-a327-a49b76896bcf"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 18:24:26.344549 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:26.344509    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9ff4b8e0-05cb-4047-a327-a49b76896bcf-cacerts" (OuterVolumeSpecName: "cacerts") pod "9ff4b8e0-05cb-4047-a327-a49b76896bcf" (UID: "9ff4b8e0-05cb-4047-a327-a49b76896bcf"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 18:24:26.344621 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:26.344602    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9ff4b8e0-05cb-4047-a327-a49b76896bcf-istio-token" (OuterVolumeSpecName: "istio-token") pod "9ff4b8e0-05cb-4047-a327-a49b76896bcf" (UID: "9ff4b8e0-05cb-4047-a327-a49b76896bcf"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 18:24:26.344657 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:26.344615    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ff4b8e0-05cb-4047-a327-a49b76896bcf-local-certs" (OuterVolumeSpecName: "local-certs") pod "9ff4b8e0-05cb-4047-a327-a49b76896bcf" (UID: "9ff4b8e0-05cb-4047-a327-a49b76896bcf"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 18:24:26.344657 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:26.344605    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9ff4b8e0-05cb-4047-a327-a49b76896bcf-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "9ff4b8e0-05cb-4047-a327-a49b76896bcf" (UID: "9ff4b8e0-05cb-4047-a327-a49b76896bcf"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 18:24:26.443315 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:26.443228    2580 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/9ff4b8e0-05cb-4047-a327-a49b76896bcf-istio-csr-dns-cert\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:24:26.443315 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:26.443262    2580 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9ff4b8e0-05cb-4047-a327-a49b76896bcf-istio-kubeconfig\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:24:26.443315 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:26.443272    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lbdgn\" (UniqueName: \"kubernetes.io/projected/9ff4b8e0-05cb-4047-a327-a49b76896bcf-kube-api-access-lbdgn\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:24:26.443315 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:26.443282    2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9ff4b8e0-05cb-4047-a327-a49b76896bcf-istio-token\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:24:26.443315 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:26.443291    2580 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/9ff4b8e0-05cb-4047-a327-a49b76896bcf-istio-csr-ca-configmap\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:24:26.443315 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:26.443300    2580 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/9ff4b8e0-05cb-4047-a327-a49b76896bcf-cacerts\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:24:26.443315 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:26.443309    2580 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/9ff4b8e0-05cb-4047-a327-a49b76896bcf-local-certs\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:24:26.868705 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:26.868672    2580 generic.go:356] "Generic (PLEG): container finished" podID="9ff4b8e0-05cb-4047-a327-a49b76896bcf" containerID="f800fb53b94784a4635655668dc6dfcd0eac68a211d24bc99e5d36219cd57790" exitCode=0
Jul 06 18:24:26.869103 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:26.868719    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-64rch"
Jul 06 18:24:26.869103 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:26.868761    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-64rch" event={"ID":"9ff4b8e0-05cb-4047-a327-a49b76896bcf","Type":"ContainerDied","Data":"f800fb53b94784a4635655668dc6dfcd0eac68a211d24bc99e5d36219cd57790"}
Jul 06 18:24:26.869103 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:26.868815    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-64rch" event={"ID":"9ff4b8e0-05cb-4047-a327-a49b76896bcf","Type":"ContainerDied","Data":"11966cc99b630ed8a1314564e879eed909e969983f4949bc466bc37e22b05e04"}
Jul 06 18:24:26.869103 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:26.868836    2580 scope.go:117] "RemoveContainer" containerID="f800fb53b94784a4635655668dc6dfcd0eac68a211d24bc99e5d36219cd57790"
Jul 06 18:24:26.877845 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:26.877827    2580 scope.go:117] "RemoveContainer" containerID="f800fb53b94784a4635655668dc6dfcd0eac68a211d24bc99e5d36219cd57790"
Jul 06 18:24:26.878160 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:24:26.878137    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f800fb53b94784a4635655668dc6dfcd0eac68a211d24bc99e5d36219cd57790\": container with ID starting with f800fb53b94784a4635655668dc6dfcd0eac68a211d24bc99e5d36219cd57790 not found: ID does not exist" containerID="f800fb53b94784a4635655668dc6dfcd0eac68a211d24bc99e5d36219cd57790"
Jul 06 18:24:26.878225 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:26.878167    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f800fb53b94784a4635655668dc6dfcd0eac68a211d24bc99e5d36219cd57790"} err="failed to get container status \"f800fb53b94784a4635655668dc6dfcd0eac68a211d24bc99e5d36219cd57790\": rpc error: code = NotFound desc = could not find container \"f800fb53b94784a4635655668dc6dfcd0eac68a211d24bc99e5d36219cd57790\": container with ID starting with f800fb53b94784a4635655668dc6dfcd0eac68a211d24bc99e5d36219cd57790 not found: ID does not exist"
Jul 06 18:24:26.891954 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:26.891924    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-64rch"]
Jul 06 18:24:26.895753 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:26.895727    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-64rch"]
Jul 06 18:24:27.322838 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:27.322815    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9ff4b8e0-05cb-4047-a327-a49b76896bcf" path="/var/lib/kubelet/pods/9ff4b8e0-05cb-4047-a327-a49b76896bcf/volumes"
Jul 06 18:24:29.883684 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:29.883649    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-c977q" event={"ID":"dfe46fe3-2f4d-4855-97c1-38e4e974c6f8","Type":"ContainerStarted","Data":"c08a3df1b1ed98b8807d84ff4649d219eac6e908bde7b4c8387d98fb79c010e0"}
Jul 06 18:24:29.884164 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:29.883724    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-c977q"
Jul 06 18:24:29.885161 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:29.885138    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4vm88" event={"ID":"e9cabd49-cdd7-4882-afc0-da156d82c239","Type":"ContainerStarted","Data":"1cb12fc896b801fbae3e09b12a248bee1b331df7d1becb3057686466471f2f64"}
Jul 06 18:24:29.885283 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:29.885267    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4vm88"
Jul 06 18:24:29.902185 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:29.902129    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-c977q" podStartSLOduration=1.9536986509999998 podStartE2EDuration="5.902114153s" totalImagesPullingTime="3.948415502s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:24:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:24:29.900468007 +0000 UTC m=+405.057866494" watchObservedRunningTime="2026-07-06 18:24:29.902114153 +0000 UTC m=+405.059512638"
Jul 06 18:24:29.918516 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:29.918460    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4vm88" podStartSLOduration=1.460773286 podStartE2EDuration="5.918443884s" totalImagesPullingTime="4.457670598s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:24:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:24:29.916856842 +0000 UTC m=+405.074255327" watchObservedRunningTime="2026-07-06 18:24:29.918443884 +0000 UTC m=+405.075842370"
Jul 06 18:24:30.832074 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:30.832034    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-cjcm8"
Jul 06 18:24:30.834539 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:24:30.834513    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-rt527"
Jul 06 18:25:00.891486 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:25:00.891453    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4vm88"
Jul 06 18:25:00.894446 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:25:00.894428    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-c977q"
Jul 06 18:25:13.986135 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:25:13.986046    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-x26cg"]
Jul 06 18:25:13.986513 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:25:13.986346    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ff4b8e0-05cb-4047-a327-a49b76896bcf" containerName="discovery"
Jul 06 18:25:13.986513 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:25:13.986356    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ff4b8e0-05cb-4047-a327-a49b76896bcf" containerName="discovery"
Jul 06 18:25:13.986513 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:25:13.986421    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="9ff4b8e0-05cb-4047-a327-a49b76896bcf" containerName="discovery"
Jul 06 18:25:13.988355 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:25:13.988340    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-x26cg"
Jul 06 18:25:13.991319 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:25:13.991298    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 06 18:25:13.991401 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:25:13.991349    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 06 18:25:13.992547 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:25:13.992533    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-kk787\""
Jul 06 18:25:13.997606 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:25:13.997573    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-x26cg"]
Jul 06 18:25:14.136185 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:25:14.136151    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rpp25\" (UniqueName: \"kubernetes.io/projected/97e8dc90-268c-48fd-a094-e8e4d86f3ed4-kube-api-access-rpp25\") pod \"kuadrant-operator-catalog-x26cg\" (UID: \"97e8dc90-268c-48fd-a094-e8e4d86f3ed4\") " pod="kuadrant-system/kuadrant-operator-catalog-x26cg"
Jul 06 18:25:14.237307 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:25:14.237210    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rpp25\" (UniqueName: \"kubernetes.io/projected/97e8dc90-268c-48fd-a094-e8e4d86f3ed4-kube-api-access-rpp25\") pod \"kuadrant-operator-catalog-x26cg\" (UID: \"97e8dc90-268c-48fd-a094-e8e4d86f3ed4\") " pod="kuadrant-system/kuadrant-operator-catalog-x26cg"
Jul 06 18:25:14.253542 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:25:14.253509    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rpp25\" (UniqueName: \"kubernetes.io/projected/97e8dc90-268c-48fd-a094-e8e4d86f3ed4-kube-api-access-rpp25\") pod \"kuadrant-operator-catalog-x26cg\" (UID: \"97e8dc90-268c-48fd-a094-e8e4d86f3ed4\") " pod="kuadrant-system/kuadrant-operator-catalog-x26cg"
Jul 06 18:25:14.298559 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:25:14.298522    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-x26cg"
Jul 06 18:25:14.350151 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:25:14.350123    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-x26cg"]
Jul 06 18:25:14.427695 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:25:14.427668    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-x26cg"]
Jul 06 18:25:14.430064 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:25:14.430035    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97e8dc90_268c_48fd_a094_e8e4d86f3ed4.slice/crio-a22eab4d8efbdac4066824ac3b1b4552cc95ca5b096c89f8dfc39a9a6c66baca WatchSource:0}: Error finding container a22eab4d8efbdac4066824ac3b1b4552cc95ca5b096c89f8dfc39a9a6c66baca: Status 404 returned error can't find the container with id a22eab4d8efbdac4066824ac3b1b4552cc95ca5b096c89f8dfc39a9a6c66baca
Jul 06 18:25:14.564144 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:25:14.564055    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-xm6rr"]
Jul 06 18:25:14.567046 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:25:14.567008    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-xm6rr"
Jul 06 18:25:14.575810 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:25:14.575777    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-xm6rr"]
Jul 06 18:25:14.743227 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:25:14.743188    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x6g9f\" (UniqueName: \"kubernetes.io/projected/37a41edd-6d01-4bdc-b374-9cb0ed5a23ac-kube-api-access-x6g9f\") pod \"kuadrant-operator-catalog-xm6rr\" (UID: \"37a41edd-6d01-4bdc-b374-9cb0ed5a23ac\") " pod="kuadrant-system/kuadrant-operator-catalog-xm6rr"
Jul 06 18:25:14.844472 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:25:14.844428    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x6g9f\" (UniqueName: \"kubernetes.io/projected/37a41edd-6d01-4bdc-b374-9cb0ed5a23ac-kube-api-access-x6g9f\") pod \"kuadrant-operator-catalog-xm6rr\" (UID: \"37a41edd-6d01-4bdc-b374-9cb0ed5a23ac\") " pod="kuadrant-system/kuadrant-operator-catalog-xm6rr"
Jul 06 18:25:14.854061 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:25:14.854030    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x6g9f\" (UniqueName: \"kubernetes.io/projected/37a41edd-6d01-4bdc-b374-9cb0ed5a23ac-kube-api-access-x6g9f\") pod \"kuadrant-operator-catalog-xm6rr\" (UID: \"37a41edd-6d01-4bdc-b374-9cb0ed5a23ac\") " pod="kuadrant-system/kuadrant-operator-catalog-xm6rr"
Jul 06 18:25:14.877584 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:25:14.877542    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-xm6rr"
Jul 06 18:25:15.027331 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:25:15.027297    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-xm6rr"]
Jul 06 18:25:15.030822 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:25:15.030792    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37a41edd_6d01_4bdc_b374_9cb0ed5a23ac.slice/crio-7a79d1b2abb5cbca3e97098c531e9d20dea5050088f707741c40a856efb91894 WatchSource:0}: Error finding container 7a79d1b2abb5cbca3e97098c531e9d20dea5050088f707741c40a856efb91894: Status 404 returned error can't find the container with id 7a79d1b2abb5cbca3e97098c531e9d20dea5050088f707741c40a856efb91894
Jul 06 18:25:15.061687 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:25:15.061654    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-x26cg" event={"ID":"97e8dc90-268c-48fd-a094-e8e4d86f3ed4","Type":"ContainerStarted","Data":"a22eab4d8efbdac4066824ac3b1b4552cc95ca5b096c89f8dfc39a9a6c66baca"}
Jul 06 18:25:15.062948 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:25:15.062925    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-xm6rr" event={"ID":"37a41edd-6d01-4bdc-b374-9cb0ed5a23ac","Type":"ContainerStarted","Data":"7a79d1b2abb5cbca3e97098c531e9d20dea5050088f707741c40a856efb91894"}
Jul 06 18:25:17.072796 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:25:17.072690    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-x26cg" event={"ID":"97e8dc90-268c-48fd-a094-e8e4d86f3ed4","Type":"ContainerStarted","Data":"6f1d6a3c14f86ee095fd9bcf8bed4062078f7ab089245be82b3e59e387df5eff"}
Jul 06 18:25:17.072796 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:25:17.072740    2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-x26cg" podUID="97e8dc90-268c-48fd-a094-e8e4d86f3ed4" containerName="registry-server" containerID="cri-o://6f1d6a3c14f86ee095fd9bcf8bed4062078f7ab089245be82b3e59e387df5eff" gracePeriod=2
Jul 06 18:25:17.074177 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:25:17.074144    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-xm6rr" event={"ID":"37a41edd-6d01-4bdc-b374-9cb0ed5a23ac","Type":"ContainerStarted","Data":"407e477143c8fb378aedc14a5ded69a73f41af3ba25b616712d7e564d9d7d147"}
Jul 06 18:25:17.092061 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:25:17.091990    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-x26cg" podStartSLOduration=1.827558365 podStartE2EDuration="4.091975835s" totalImagesPullingTime="2.26441747s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:25:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:25:17.090328154 +0000 UTC m=+452.247726641" watchObservedRunningTime="2026-07-06 18:25:17.091975835 +0000 UTC m=+452.249374321"
Jul 06 18:25:17.106494 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:25:17.106442    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-xm6rr" podStartSLOduration=1.440430134 podStartE2EDuration="3.106426497s" totalImagesPullingTime="1.665996363s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:25:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:25:17.10563493 +0000 UTC m=+452.263033449" watchObservedRunningTime="2026-07-06 18:25:17.106426497 +0000 UTC m=+452.263824983"
Jul 06 18:25:17.308997 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:25:17.308968    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-x26cg"
Jul 06 18:25:17.467264 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:25:17.467174    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rpp25\" (UniqueName: \"kubernetes.io/projected/97e8dc90-268c-48fd-a094-e8e4d86f3ed4-kube-api-access-rpp25\") pod \"97e8dc90-268c-48fd-a094-e8e4d86f3ed4\" (UID: \"97e8dc90-268c-48fd-a094-e8e4d86f3ed4\") "
Jul 06 18:25:17.469364 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:25:17.469335    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/97e8dc90-268c-48fd-a094-e8e4d86f3ed4-kube-api-access-rpp25" (OuterVolumeSpecName: "kube-api-access-rpp25") pod "97e8dc90-268c-48fd-a094-e8e4d86f3ed4" (UID: "97e8dc90-268c-48fd-a094-e8e4d86f3ed4"). InnerVolumeSpecName "kube-api-access-rpp25". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 18:25:17.568257 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:25:17.568221    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rpp25\" (UniqueName: \"kubernetes.io/projected/97e8dc90-268c-48fd-a094-e8e4d86f3ed4-kube-api-access-rpp25\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:25:18.079283 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:25:18.079245    2580 generic.go:356] "Generic (PLEG): container finished" podID="97e8dc90-268c-48fd-a094-e8e4d86f3ed4" containerID="6f1d6a3c14f86ee095fd9bcf8bed4062078f7ab089245be82b3e59e387df5eff" exitCode=0
Jul 06 18:25:18.079665 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:25:18.079290    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-x26cg"
Jul 06 18:25:18.079665 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:25:18.079332    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-x26cg" event={"ID":"97e8dc90-268c-48fd-a094-e8e4d86f3ed4","Type":"ContainerDied","Data":"6f1d6a3c14f86ee095fd9bcf8bed4062078f7ab089245be82b3e59e387df5eff"}
Jul 06 18:25:18.079665 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:25:18.079379    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-x26cg" event={"ID":"97e8dc90-268c-48fd-a094-e8e4d86f3ed4","Type":"ContainerDied","Data":"a22eab4d8efbdac4066824ac3b1b4552cc95ca5b096c89f8dfc39a9a6c66baca"}
Jul 06 18:25:18.079665 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:25:18.079399    2580 scope.go:117] "RemoveContainer" containerID="6f1d6a3c14f86ee095fd9bcf8bed4062078f7ab089245be82b3e59e387df5eff"
Jul 06 18:25:18.088230 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:25:18.088211    2580 scope.go:117] "RemoveContainer" containerID="6f1d6a3c14f86ee095fd9bcf8bed4062078f7ab089245be82b3e59e387df5eff"
Jul 06 18:25:18.088488 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:25:18.088468    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6f1d6a3c14f86ee095fd9bcf8bed4062078f7ab089245be82b3e59e387df5eff\": container with ID starting with 6f1d6a3c14f86ee095fd9bcf8bed4062078f7ab089245be82b3e59e387df5eff not found: ID does not exist" containerID="6f1d6a3c14f86ee095fd9bcf8bed4062078f7ab089245be82b3e59e387df5eff"
Jul 06 18:25:18.088534 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:25:18.088497    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f1d6a3c14f86ee095fd9bcf8bed4062078f7ab089245be82b3e59e387df5eff"} err="failed to get container status \"6f1d6a3c14f86ee095fd9bcf8bed4062078f7ab089245be82b3e59e387df5eff\": rpc error: code = NotFound desc = could not find container \"6f1d6a3c14f86ee095fd9bcf8bed4062078f7ab089245be82b3e59e387df5eff\": container with ID starting with 6f1d6a3c14f86ee095fd9bcf8bed4062078f7ab089245be82b3e59e387df5eff not found: ID does not exist"
Jul 06 18:25:18.099864 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:25:18.099839    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-x26cg"]
Jul 06 18:25:18.102578 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:25:18.102560    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-x26cg"]
Jul 06 18:25:19.322081 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:25:19.322043    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="97e8dc90-268c-48fd-a094-e8e4d86f3ed4" path="/var/lib/kubelet/pods/97e8dc90-268c-48fd-a094-e8e4d86f3ed4/volumes"
Jul 06 18:25:24.877969 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:25:24.877935    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-xm6rr"
Jul 06 18:25:24.878425 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:25:24.878207    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-xm6rr"
Jul 06 18:25:24.899923 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:25:24.899892    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-xm6rr"
Jul 06 18:25:25.129520 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:25:25.129455    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-xm6rr"
Jul 06 18:25:47.766069 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:25:47.766033    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m4rgl"]
Jul 06 18:25:47.766485 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:25:47.766327    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="97e8dc90-268c-48fd-a094-e8e4d86f3ed4" containerName="registry-server"
Jul 06 18:25:47.766485 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:25:47.766337    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="97e8dc90-268c-48fd-a094-e8e4d86f3ed4" containerName="registry-server"
Jul 06 18:25:47.766485 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:25:47.766383    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="97e8dc90-268c-48fd-a094-e8e4d86f3ed4" containerName="registry-server"
Jul 06 18:25:47.770096 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:25:47.770078    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m4rgl"
Jul 06 18:25:47.772872 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:25:47.772853    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-v4csq\""
Jul 06 18:25:47.785167 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:25:47.785144    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m4rgl"]
Jul 06 18:25:47.787904 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:25:47.787882    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-476q9\" (UniqueName: \"kubernetes.io/projected/169ad069-4a3e-4909-800a-86927ad21092-kube-api-access-476q9\") pod \"kuadrant-operator-controller-manager-8684c9ff49-m4rgl\" (UID: \"169ad069-4a3e-4909-800a-86927ad21092\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m4rgl"
Jul 06 18:25:47.787987 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:25:47.787922    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/169ad069-4a3e-4909-800a-86927ad21092-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-m4rgl\" (UID: \"169ad069-4a3e-4909-800a-86927ad21092\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m4rgl"
Jul 06 18:25:47.888428 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:25:47.888389    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-476q9\" (UniqueName: \"kubernetes.io/projected/169ad069-4a3e-4909-800a-86927ad21092-kube-api-access-476q9\") pod \"kuadrant-operator-controller-manager-8684c9ff49-m4rgl\" (UID: \"169ad069-4a3e-4909-800a-86927ad21092\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m4rgl"
Jul 06 18:25:47.888606 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:25:47.888452    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/169ad069-4a3e-4909-800a-86927ad21092-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-m4rgl\" (UID: \"169ad069-4a3e-4909-800a-86927ad21092\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m4rgl"
Jul 06 18:25:47.888831 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:25:47.888809    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/169ad069-4a3e-4909-800a-86927ad21092-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-m4rgl\" (UID: \"169ad069-4a3e-4909-800a-86927ad21092\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m4rgl"
Jul 06 18:25:47.897062 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:25:47.897034    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-476q9\" (UniqueName: \"kubernetes.io/projected/169ad069-4a3e-4909-800a-86927ad21092-kube-api-access-476q9\") pod \"kuadrant-operator-controller-manager-8684c9ff49-m4rgl\" (UID: \"169ad069-4a3e-4909-800a-86927ad21092\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m4rgl"
Jul 06 18:25:48.079703 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:25:48.079669    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m4rgl"
Jul 06 18:25:48.213283 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:25:48.213257    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m4rgl"]
Jul 06 18:25:48.215695 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:25:48.215664    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod169ad069_4a3e_4909_800a_86927ad21092.slice/crio-5b1a479a5debe4cf8d27091661c49649d9ef7278ba9d960da5441b0df3196815 WatchSource:0}: Error finding container 5b1a479a5debe4cf8d27091661c49649d9ef7278ba9d960da5441b0df3196815: Status 404 returned error can't find the container with id 5b1a479a5debe4cf8d27091661c49649d9ef7278ba9d960da5441b0df3196815
Jul 06 18:25:49.204333 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:25:49.204277    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m4rgl" event={"ID":"169ad069-4a3e-4909-800a-86927ad21092","Type":"ContainerStarted","Data":"5b1a479a5debe4cf8d27091661c49649d9ef7278ba9d960da5441b0df3196815"}
Jul 06 18:25:50.981002 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:25:50.980960    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-px77q"]
Jul 06 18:25:50.983349 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:25:50.983326    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-px77q"
Jul 06 18:25:50.986076 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:25:50.986047    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-76pxg\""
Jul 06 18:25:50.986203 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:25:50.986186    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Jul 06 18:25:50.999031 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:25:50.998989    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-px77q"]
Jul 06 18:25:51.016790 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:25:51.016767    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k6tbh\" (UniqueName: \"kubernetes.io/projected/d8f47480-21bb-4696-b306-c0b5099e9f0e-kube-api-access-k6tbh\") pod \"dns-operator-controller-manager-864dd78c4c-px77q\" (UID: \"d8f47480-21bb-4696-b306-c0b5099e9f0e\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-px77q"
Jul 06 18:25:51.117541 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:25:51.117502    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k6tbh\" (UniqueName: \"kubernetes.io/projected/d8f47480-21bb-4696-b306-c0b5099e9f0e-kube-api-access-k6tbh\") pod \"dns-operator-controller-manager-864dd78c4c-px77q\" (UID: \"d8f47480-21bb-4696-b306-c0b5099e9f0e\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-px77q"
Jul 06 18:25:51.131746 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:25:51.131715    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k6tbh\" (UniqueName: \"kubernetes.io/projected/d8f47480-21bb-4696-b306-c0b5099e9f0e-kube-api-access-k6tbh\") pod \"dns-operator-controller-manager-864dd78c4c-px77q\" (UID: \"d8f47480-21bb-4696-b306-c0b5099e9f0e\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-px77q"
Jul 06 18:25:51.293213 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:25:51.293129    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-px77q"
Jul 06 18:25:51.463069 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:25:51.463042    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-px77q"]
Jul 06 18:25:51.466065 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:25:51.466032    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd8f47480_21bb_4696_b306_c0b5099e9f0e.slice/crio-0de047b4603ae81c10b6ecd9859673f710f126b42a9e66491edc23c743b0c986 WatchSource:0}: Error finding container 0de047b4603ae81c10b6ecd9859673f710f126b42a9e66491edc23c743b0c986: Status 404 returned error can't find the container with id 0de047b4603ae81c10b6ecd9859673f710f126b42a9e66491edc23c743b0c986
Jul 06 18:25:52.219585 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:25:52.219548    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-px77q" event={"ID":"d8f47480-21bb-4696-b306-c0b5099e9f0e","Type":"ContainerStarted","Data":"0de047b4603ae81c10b6ecd9859673f710f126b42a9e66491edc23c743b0c986"}
Jul 06 18:25:54.077616 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:25:54.077582    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hpv8d"]
Jul 06 18:25:54.080991 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:25:54.080969    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hpv8d"
Jul 06 18:25:54.083702 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:25:54.083674    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-5hcr5\""
Jul 06 18:25:54.089919 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:25:54.089889    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hpv8d"]
Jul 06 18:25:54.142476 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:25:54.142436    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dw8z7\" (UniqueName: \"kubernetes.io/projected/9b5cebf5-1671-4635-ab43-31f760286b43-kube-api-access-dw8z7\") pod \"limitador-operator-controller-manager-859d7b78bd-hpv8d\" (UID: \"9b5cebf5-1671-4635-ab43-31f760286b43\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hpv8d"
Jul 06 18:25:54.243204 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:25:54.243148    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dw8z7\" (UniqueName: \"kubernetes.io/projected/9b5cebf5-1671-4635-ab43-31f760286b43-kube-api-access-dw8z7\") pod \"limitador-operator-controller-manager-859d7b78bd-hpv8d\" (UID: \"9b5cebf5-1671-4635-ab43-31f760286b43\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hpv8d"
Jul 06 18:25:54.252229 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:25:54.252201    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dw8z7\" (UniqueName: \"kubernetes.io/projected/9b5cebf5-1671-4635-ab43-31f760286b43-kube-api-access-dw8z7\") pod \"limitador-operator-controller-manager-859d7b78bd-hpv8d\" (UID: \"9b5cebf5-1671-4635-ab43-31f760286b43\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hpv8d"
Jul 06 18:25:54.394851 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:25:54.394759    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hpv8d"
Jul 06 18:25:55.117655 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:25:55.117607    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hpv8d"]
Jul 06 18:25:55.123103 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:25:55.123075    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b5cebf5_1671_4635_ab43_31f760286b43.slice/crio-ef69e8c125133a81f07c877edcfbbd3a96352e04beaa4799ce053853b9bebcad WatchSource:0}: Error finding container ef69e8c125133a81f07c877edcfbbd3a96352e04beaa4799ce053853b9bebcad: Status 404 returned error can't find the container with id ef69e8c125133a81f07c877edcfbbd3a96352e04beaa4799ce053853b9bebcad
Jul 06 18:25:55.236966 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:25:55.236869    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-px77q" event={"ID":"d8f47480-21bb-4696-b306-c0b5099e9f0e","Type":"ContainerStarted","Data":"e4802fbb6a7c9330b1c95620180295d844a219a20b8ee6efc420022731edb847"}
Jul 06 18:25:55.237405 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:25:55.237226    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-px77q"
Jul 06 18:25:55.238866 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:25:55.238837    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hpv8d" event={"ID":"9b5cebf5-1671-4635-ab43-31f760286b43","Type":"ContainerStarted","Data":"ef69e8c125133a81f07c877edcfbbd3a96352e04beaa4799ce053853b9bebcad"}
Jul 06 18:25:55.243176 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:25:55.243145    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m4rgl" event={"ID":"169ad069-4a3e-4909-800a-86927ad21092","Type":"ContainerStarted","Data":"bef8796379eb2c3878cc9ea9ca591b2e9ef671883c35298372f40f667a6d1b00"}
Jul 06 18:25:55.243291 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:25:55.243282    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m4rgl"
Jul 06 18:25:55.267542 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:25:55.267490    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-px77q" podStartSLOduration=1.7881371640000001 podStartE2EDuration="5.267474203s" totalImagesPullingTime="3.479337039s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:25:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:25:55.267234979 +0000 UTC m=+490.424633505" watchObservedRunningTime="2026-07-06 18:25:55.267474203 +0000 UTC m=+490.424872691"
Jul 06 18:25:55.290642 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:25:55.290590    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m4rgl" podStartSLOduration=1.548996131 podStartE2EDuration="8.29057458s" totalImagesPullingTime="6.741578449s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:25:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:25:55.289760603 +0000 UTC m=+490.447159123" watchObservedRunningTime="2026-07-06 18:25:55.29057458 +0000 UTC m=+490.447973128"
Jul 06 18:25:58.097656 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:25:58.097623    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-l47mq"]
Jul 06 18:25:58.100833 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:25:58.100816    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-l47mq"
Jul 06 18:25:58.103659 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:25:58.103629    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"authorino-operator-75cc8d5cb7-l47mq\" is forbidden: User \"system:node:ip-10-0-134-113.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-113.ec2.internal' and this object" podUID="5a1328d5-22f7-4276-a5b5-506f97578629" pod="kuadrant-system/authorino-operator-75cc8d5cb7-l47mq"
Jul 06 18:25:58.103763 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:25:58.103659    2580 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"authorino-operator-dockercfg-t5v57\" is forbidden: User \"system:node:ip-10-0-134-113.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-113.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-t5v57\"" type="*v1.Secret"
Jul 06 18:25:58.120183 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:25:58.120155    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-l47mq"]
Jul 06 18:25:58.176276 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:25:58.176240    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-42qr2\" (UniqueName: \"kubernetes.io/projected/5a1328d5-22f7-4276-a5b5-506f97578629-kube-api-access-42qr2\") pod \"authorino-operator-75cc8d5cb7-l47mq\" (UID: \"5a1328d5-22f7-4276-a5b5-506f97578629\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-l47mq"
Jul 06 18:25:58.257227 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:25:58.257182    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hpv8d" event={"ID":"9b5cebf5-1671-4635-ab43-31f760286b43","Type":"ContainerStarted","Data":"c3382b4bb87ddc67816f62340b78014e63083abe01c7b96f1131861c29b8abc6"}
Jul 06 18:25:58.257415 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:25:58.257282    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hpv8d"
Jul 06 18:25:58.277056 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:25:58.276991    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-42qr2\" (UniqueName: \"kubernetes.io/projected/5a1328d5-22f7-4276-a5b5-506f97578629-kube-api-access-42qr2\") pod \"authorino-operator-75cc8d5cb7-l47mq\" (UID: \"5a1328d5-22f7-4276-a5b5-506f97578629\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-l47mq"
Jul 06 18:25:58.278135 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:25:58.278092    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hpv8d" podStartSLOduration=2.116996686 podStartE2EDuration="4.278077095s" totalImagesPullingTime="2.161080409s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:25:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:25:58.276971635 +0000 UTC m=+493.434370112" watchObservedRunningTime="2026-07-06 18:25:58.278077095 +0000 UTC m=+493.435475580"
Jul 06 18:25:58.289264 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:25:58.289236    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-42qr2\" (UniqueName: \"kubernetes.io/projected/5a1328d5-22f7-4276-a5b5-506f97578629-kube-api-access-42qr2\") pod \"authorino-operator-75cc8d5cb7-l47mq\" (UID: \"5a1328d5-22f7-4276-a5b5-506f97578629\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-l47mq"
Jul 06 18:25:59.411758 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:25:59.411715    2580 kubelet_pods.go:1082] "Unable to retrieve pull secret, the image pull may not succeed." pod="kuadrant-system/authorino-operator-75cc8d5cb7-l47mq" secret="" err="failed to sync secret cache: timed out waiting for the condition"
Jul 06 18:25:59.412206 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:25:59.411788    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-l47mq"
Jul 06 18:25:59.554092 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:25:59.554053    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-t5v57\""
Jul 06 18:25:59.589559 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:25:59.589528    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-l47mq"]
Jul 06 18:25:59.596842 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:25:59.596819    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a1328d5_22f7_4276_a5b5_506f97578629.slice/crio-0a1b52d934f785990668e56b27495d92290a01a3af898afbb0dc305437faee3c WatchSource:0}: Error finding container 0a1b52d934f785990668e56b27495d92290a01a3af898afbb0dc305437faee3c: Status 404 returned error can't find the container with id 0a1b52d934f785990668e56b27495d92290a01a3af898afbb0dc305437faee3c
Jul 06 18:26:00.266609 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:00.266571    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-l47mq" event={"ID":"5a1328d5-22f7-4276-a5b5-506f97578629","Type":"ContainerStarted","Data":"0a1b52d934f785990668e56b27495d92290a01a3af898afbb0dc305437faee3c"}
Jul 06 18:26:03.282838 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:03.282804    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-l47mq" event={"ID":"5a1328d5-22f7-4276-a5b5-506f97578629","Type":"ContainerStarted","Data":"22099678642e31421bbe4dabdd46e37867698b85d66f63e1c498d2c911326cff"}
Jul 06 18:26:03.283326 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:03.282929    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-l47mq"
Jul 06 18:26:03.300366 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:03.300312    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-l47mq" podStartSLOduration=2.353942773 podStartE2EDuration="5.300291505s" totalImagesPullingTime="2.946348732s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:25:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:26:03.300052496 +0000 UTC m=+498.457450983" watchObservedRunningTime="2026-07-06 18:26:03.300291505 +0000 UTC m=+498.457689985"
Jul 06 18:26:06.248805 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:06.248763    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-px77q"
Jul 06 18:26:06.249336 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:06.248850    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m4rgl"
Jul 06 18:26:07.965930 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:07.965896    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m4rgl"]
Jul 06 18:26:07.966405 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:07.966103    2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m4rgl" podUID="169ad069-4a3e-4909-800a-86927ad21092" containerName="manager" containerID="cri-o://bef8796379eb2c3878cc9ea9ca591b2e9ef671883c35298372f40f667a6d1b00" gracePeriod=2
Jul 06 18:26:07.972848 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:07.972817    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m4rgl"]
Jul 06 18:26:07.974465 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:26:07.974430    2580 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"169ad069-4a3e-4909-800a-86927ad21092\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-8684c9ff49-m4rgl\": pods \"kuadrant-operator-controller-manager-8684c9ff49-m4rgl\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m4rgl"
Jul 06 18:26:07.987417 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:07.987391    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hpv8d"]
Jul 06 18:26:07.987735 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:07.987623    2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hpv8d" podUID="9b5cebf5-1671-4635-ab43-31f760286b43" containerName="manager" containerID="cri-o://c3382b4bb87ddc67816f62340b78014e63083abe01c7b96f1131861c29b8abc6" gracePeriod=2
Jul 06 18:26:07.990119 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:07.989837    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hpv8d"
Jul 06 18:26:07.990246 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:26:07.990127    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-m4rgl\" is forbidden: User \"system:node:ip-10-0-134-113.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-113.ec2.internal' and this object" podUID="169ad069-4a3e-4909-800a-86927ad21092" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m4rgl"
Jul 06 18:26:07.992110 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:07.991991    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ccj5r"]
Jul 06 18:26:07.992480 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:07.992460    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="169ad069-4a3e-4909-800a-86927ad21092" containerName="manager"
Jul 06 18:26:07.992575 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:07.992506    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="169ad069-4a3e-4909-800a-86927ad21092" containerName="manager"
Jul 06 18:26:07.992635 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:07.992609    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="169ad069-4a3e-4909-800a-86927ad21092" containerName="manager"
Jul 06 18:26:07.996791 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:07.996765    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ccj5r"
Jul 06 18:26:07.999665 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:07.999641    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hpv8d"]
Jul 06 18:26:08.010539 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:26:08.010502    2580 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"9b5cebf5-1671-4635-ab43-31f760286b43\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"limitador-operator-controller-manager-859d7b78bd-hpv8d\": pods \"limitador-operator-controller-manager-859d7b78bd-hpv8d\" not found" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hpv8d"
Jul 06 18:26:08.013093 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:26:08.013065    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-hpv8d\" is forbidden: User \"system:node:ip-10-0-134-113.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-113.ec2.internal' and this object" podUID="9b5cebf5-1671-4635-ab43-31f760286b43" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hpv8d"
Jul 06 18:26:08.013824 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:08.013804    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ccj5r"]
Jul 06 18:26:08.015445 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:08.015423    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hdrzx"]
Jul 06 18:26:08.015784 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:08.015763    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b5cebf5-1671-4635-ab43-31f760286b43" containerName="manager"
Jul 06 18:26:08.015784 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:08.015783    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b5cebf5-1671-4635-ab43-31f760286b43" containerName="manager"
Jul 06 18:26:08.015925 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:08.015861    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="9b5cebf5-1671-4635-ab43-31f760286b43" containerName="manager"
Jul 06 18:26:08.019500 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:08.019483    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hdrzx"
Jul 06 18:26:08.028680 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:08.028656    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hdrzx"]
Jul 06 18:26:08.033598 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:26:08.033568    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-m4rgl\" is forbidden: User \"system:node:ip-10-0-134-113.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-113.ec2.internal' and this object" podUID="169ad069-4a3e-4909-800a-86927ad21092" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m4rgl"
Jul 06 18:26:08.054577 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:26:08.054543    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-hpv8d\" is forbidden: User \"system:node:ip-10-0-134-113.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-113.ec2.internal' and this object" podUID="9b5cebf5-1671-4635-ab43-31f760286b43" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hpv8d"
Jul 06 18:26:08.056891 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:26:08.056852    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-m4rgl\" is forbidden: User \"system:node:ip-10-0-134-113.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-113.ec2.internal' and this object" podUID="169ad069-4a3e-4909-800a-86927ad21092" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m4rgl"
Jul 06 18:26:08.064246 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:08.064217    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t2cl9\" (UniqueName: \"kubernetes.io/projected/177ab09a-cc55-4f33-a7f4-257872b8da05-kube-api-access-t2cl9\") pod \"kuadrant-operator-controller-manager-8684c9ff49-ccj5r\" (UID: \"177ab09a-cc55-4f33-a7f4-257872b8da05\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ccj5r"
Jul 06 18:26:08.064344 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:08.064255    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/177ab09a-cc55-4f33-a7f4-257872b8da05-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-ccj5r\" (UID: \"177ab09a-cc55-4f33-a7f4-257872b8da05\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ccj5r"
Jul 06 18:26:08.064344 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:08.064302    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xskhv\" (UniqueName: \"kubernetes.io/projected/5f34ac69-ec56-4482-a77a-d60caac43ac0-kube-api-access-xskhv\") pod \"limitador-operator-controller-manager-859d7b78bd-hdrzx\" (UID: \"5f34ac69-ec56-4482-a77a-d60caac43ac0\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hdrzx"
Jul 06 18:26:08.165951 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:08.165572    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t2cl9\" (UniqueName: \"kubernetes.io/projected/177ab09a-cc55-4f33-a7f4-257872b8da05-kube-api-access-t2cl9\") pod \"kuadrant-operator-controller-manager-8684c9ff49-ccj5r\" (UID: \"177ab09a-cc55-4f33-a7f4-257872b8da05\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ccj5r"
Jul 06 18:26:08.165951 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:08.165634    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/177ab09a-cc55-4f33-a7f4-257872b8da05-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-ccj5r\" (UID: \"177ab09a-cc55-4f33-a7f4-257872b8da05\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ccj5r"
Jul 06 18:26:08.165951 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:08.165722    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xskhv\" (UniqueName: \"kubernetes.io/projected/5f34ac69-ec56-4482-a77a-d60caac43ac0-kube-api-access-xskhv\") pod \"limitador-operator-controller-manager-859d7b78bd-hdrzx\" (UID: \"5f34ac69-ec56-4482-a77a-d60caac43ac0\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hdrzx"
Jul 06 18:26:08.166220 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:08.166121    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/177ab09a-cc55-4f33-a7f4-257872b8da05-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-ccj5r\" (UID: \"177ab09a-cc55-4f33-a7f4-257872b8da05\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ccj5r"
Jul 06 18:26:08.174872 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:08.174844    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t2cl9\" (UniqueName: \"kubernetes.io/projected/177ab09a-cc55-4f33-a7f4-257872b8da05-kube-api-access-t2cl9\") pod \"kuadrant-operator-controller-manager-8684c9ff49-ccj5r\" (UID: \"177ab09a-cc55-4f33-a7f4-257872b8da05\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ccj5r"
Jul 06 18:26:08.175667 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:08.175642    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xskhv\" (UniqueName: \"kubernetes.io/projected/5f34ac69-ec56-4482-a77a-d60caac43ac0-kube-api-access-xskhv\") pod \"limitador-operator-controller-manager-859d7b78bd-hdrzx\" (UID: \"5f34ac69-ec56-4482-a77a-d60caac43ac0\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hdrzx"
Jul 06 18:26:08.228487 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:08.228464    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m4rgl"
Jul 06 18:26:08.230438 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:08.230419    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hpv8d"
Jul 06 18:26:08.230815 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:26:08.230787    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-m4rgl\" is forbidden: User \"system:node:ip-10-0-134-113.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-113.ec2.internal' and this object" podUID="169ad069-4a3e-4909-800a-86927ad21092" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m4rgl"
Jul 06 18:26:08.234384 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:26:08.234365    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-hpv8d\" is forbidden: User \"system:node:ip-10-0-134-113.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-113.ec2.internal' and this object" podUID="9b5cebf5-1671-4635-ab43-31f760286b43" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hpv8d"
Jul 06 18:26:08.236572 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:26:08.236551    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-hpv8d\" is forbidden: User \"system:node:ip-10-0-134-113.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-113.ec2.internal' and this object" podUID="9b5cebf5-1671-4635-ab43-31f760286b43" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hpv8d"
Jul 06 18:26:08.238769 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:26:08.238742    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-m4rgl\" is forbidden: User \"system:node:ip-10-0-134-113.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-113.ec2.internal' and this object" podUID="169ad069-4a3e-4909-800a-86927ad21092" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m4rgl"
Jul 06 18:26:08.304481 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:08.304449    2580 generic.go:356] "Generic (PLEG): container finished" podID="9b5cebf5-1671-4635-ab43-31f760286b43" containerID="c3382b4bb87ddc67816f62340b78014e63083abe01c7b96f1131861c29b8abc6" exitCode=0
Jul 06 18:26:08.304633 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:08.304494    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hpv8d"
Jul 06 18:26:08.304633 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:08.304556    2580 scope.go:117] "RemoveContainer" containerID="c3382b4bb87ddc67816f62340b78014e63083abe01c7b96f1131861c29b8abc6"
Jul 06 18:26:08.305778 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:08.305757    2580 generic.go:356] "Generic (PLEG): container finished" podID="169ad069-4a3e-4909-800a-86927ad21092" containerID="bef8796379eb2c3878cc9ea9ca591b2e9ef671883c35298372f40f667a6d1b00" exitCode=0
Jul 06 18:26:08.305886 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:08.305802    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m4rgl"
Jul 06 18:26:08.307120 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:26:08.307098    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-m4rgl\" is forbidden: User \"system:node:ip-10-0-134-113.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-113.ec2.internal' and this object" podUID="169ad069-4a3e-4909-800a-86927ad21092" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m4rgl"
Jul 06 18:26:08.309467 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:26:08.309430    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-hpv8d\" is forbidden: User \"system:node:ip-10-0-134-113.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-113.ec2.internal' and this object" podUID="9b5cebf5-1671-4635-ab43-31f760286b43" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hpv8d"
Jul 06 18:26:08.311473 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:26:08.311454    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-m4rgl\" is forbidden: User \"system:node:ip-10-0-134-113.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-113.ec2.internal' and this object" podUID="169ad069-4a3e-4909-800a-86927ad21092" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m4rgl"
Jul 06 18:26:08.312573 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:08.312556    2580 scope.go:117] "RemoveContainer" containerID="c3382b4bb87ddc67816f62340b78014e63083abe01c7b96f1131861c29b8abc6"
Jul 06 18:26:08.312806 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:26:08.312785    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c3382b4bb87ddc67816f62340b78014e63083abe01c7b96f1131861c29b8abc6\": container with ID starting with c3382b4bb87ddc67816f62340b78014e63083abe01c7b96f1131861c29b8abc6 not found: ID does not exist" containerID="c3382b4bb87ddc67816f62340b78014e63083abe01c7b96f1131861c29b8abc6"
Jul 06 18:26:08.312865 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:08.312813    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c3382b4bb87ddc67816f62340b78014e63083abe01c7b96f1131861c29b8abc6"} err="failed to get container status \"c3382b4bb87ddc67816f62340b78014e63083abe01c7b96f1131861c29b8abc6\": rpc error: code = NotFound desc = could not find container \"c3382b4bb87ddc67816f62340b78014e63083abe01c7b96f1131861c29b8abc6\": container with ID starting with c3382b4bb87ddc67816f62340b78014e63083abe01c7b96f1131861c29b8abc6 not found: ID does not exist"
Jul 06 18:26:08.312865 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:08.312831    2580 scope.go:117] "RemoveContainer" containerID="bef8796379eb2c3878cc9ea9ca591b2e9ef671883c35298372f40f667a6d1b00"
Jul 06 18:26:08.313516 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:26:08.313493    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-hpv8d\" is forbidden: User \"system:node:ip-10-0-134-113.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-113.ec2.internal' and this object" podUID="9b5cebf5-1671-4635-ab43-31f760286b43" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hpv8d"
Jul 06 18:26:08.319985 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:08.319965    2580 scope.go:117] "RemoveContainer" containerID="bef8796379eb2c3878cc9ea9ca591b2e9ef671883c35298372f40f667a6d1b00"
Jul 06 18:26:08.320238 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:26:08.320218    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bef8796379eb2c3878cc9ea9ca591b2e9ef671883c35298372f40f667a6d1b00\": container with ID starting with bef8796379eb2c3878cc9ea9ca591b2e9ef671883c35298372f40f667a6d1b00 not found: ID does not exist" containerID="bef8796379eb2c3878cc9ea9ca591b2e9ef671883c35298372f40f667a6d1b00"
Jul 06 18:26:08.320293 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:08.320245    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bef8796379eb2c3878cc9ea9ca591b2e9ef671883c35298372f40f667a6d1b00"} err="failed to get container status \"bef8796379eb2c3878cc9ea9ca591b2e9ef671883c35298372f40f667a6d1b00\": rpc error: code = NotFound desc = could not find container \"bef8796379eb2c3878cc9ea9ca591b2e9ef671883c35298372f40f667a6d1b00\": container with ID starting with bef8796379eb2c3878cc9ea9ca591b2e9ef671883c35298372f40f667a6d1b00 not found: ID does not exist"
Jul 06 18:26:08.367748 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:08.367711    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-476q9\" (UniqueName: \"kubernetes.io/projected/169ad069-4a3e-4909-800a-86927ad21092-kube-api-access-476q9\") pod \"169ad069-4a3e-4909-800a-86927ad21092\" (UID: \"169ad069-4a3e-4909-800a-86927ad21092\") "
Jul 06 18:26:08.367748 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:08.367752    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/169ad069-4a3e-4909-800a-86927ad21092-extensions-socket-volume\") pod \"169ad069-4a3e-4909-800a-86927ad21092\" (UID: \"169ad069-4a3e-4909-800a-86927ad21092\") "
Jul 06 18:26:08.367926 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:08.367779    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dw8z7\" (UniqueName: \"kubernetes.io/projected/9b5cebf5-1671-4635-ab43-31f760286b43-kube-api-access-dw8z7\") pod \"9b5cebf5-1671-4635-ab43-31f760286b43\" (UID: \"9b5cebf5-1671-4635-ab43-31f760286b43\") "
Jul 06 18:26:08.368281 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:08.368252    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/169ad069-4a3e-4909-800a-86927ad21092-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "169ad069-4a3e-4909-800a-86927ad21092" (UID: "169ad069-4a3e-4909-800a-86927ad21092"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 18:26:08.369845 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:08.369822    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/169ad069-4a3e-4909-800a-86927ad21092-kube-api-access-476q9" (OuterVolumeSpecName: "kube-api-access-476q9") pod "169ad069-4a3e-4909-800a-86927ad21092" (UID: "169ad069-4a3e-4909-800a-86927ad21092"). InnerVolumeSpecName "kube-api-access-476q9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 18:26:08.369845 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:08.369830    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9b5cebf5-1671-4635-ab43-31f760286b43-kube-api-access-dw8z7" (OuterVolumeSpecName: "kube-api-access-dw8z7") pod "9b5cebf5-1671-4635-ab43-31f760286b43" (UID: "9b5cebf5-1671-4635-ab43-31f760286b43"). InnerVolumeSpecName "kube-api-access-dw8z7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 18:26:08.405645 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:08.405614    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ccj5r"
Jul 06 18:26:08.412767 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:08.412740    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hdrzx"
Jul 06 18:26:08.468956 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:08.468599    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-476q9\" (UniqueName: \"kubernetes.io/projected/169ad069-4a3e-4909-800a-86927ad21092-kube-api-access-476q9\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:26:08.468956 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:08.468635    2580 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/169ad069-4a3e-4909-800a-86927ad21092-extensions-socket-volume\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:26:08.468956 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:08.468653    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dw8z7\" (UniqueName: \"kubernetes.io/projected/9b5cebf5-1671-4635-ab43-31f760286b43-kube-api-access-dw8z7\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:26:08.552814 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:08.552756    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ccj5r"]
Jul 06 18:26:08.555615 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:26:08.555587    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod177ab09a_cc55_4f33_a7f4_257872b8da05.slice/crio-015f379447da842a95bbf7197911414476a4b6965878b503b5e153b9c97cd138 WatchSource:0}: Error finding container 015f379447da842a95bbf7197911414476a4b6965878b503b5e153b9c97cd138: Status 404 returned error can't find the container with id 015f379447da842a95bbf7197911414476a4b6965878b503b5e153b9c97cd138
Jul 06 18:26:08.566454 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:08.566433    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hdrzx"]
Jul 06 18:26:08.568562 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:26:08.568542    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f34ac69_ec56_4482_a77a_d60caac43ac0.slice/crio-782d420d434c9b8c94d88284e6e5fb921c6de6111dadf09f69acb2d9f34df463 WatchSource:0}: Error finding container 782d420d434c9b8c94d88284e6e5fb921c6de6111dadf09f69acb2d9f34df463: Status 404 returned error can't find the container with id 782d420d434c9b8c94d88284e6e5fb921c6de6111dadf09f69acb2d9f34df463
Jul 06 18:26:08.618304 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:26:08.618274    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-m4rgl\" is forbidden: User \"system:node:ip-10-0-134-113.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-113.ec2.internal' and this object" podUID="169ad069-4a3e-4909-800a-86927ad21092" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m4rgl"
Jul 06 18:26:08.620432 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:26:08.620409    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-hpv8d\" is forbidden: User \"system:node:ip-10-0-134-113.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-113.ec2.internal' and this object" podUID="9b5cebf5-1671-4635-ab43-31f760286b43" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hpv8d"
Jul 06 18:26:08.622411 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:26:08.622385    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-hpv8d\" is forbidden: User \"system:node:ip-10-0-134-113.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-113.ec2.internal' and this object" podUID="9b5cebf5-1671-4635-ab43-31f760286b43" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hpv8d"
Jul 06 18:26:08.626773 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:26:08.626748    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-m4rgl\" is forbidden: User \"system:node:ip-10-0-134-113.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-113.ec2.internal' and this object" podUID="169ad069-4a3e-4909-800a-86927ad21092" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m4rgl"
Jul 06 18:26:09.313122 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:09.313035    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ccj5r" event={"ID":"177ab09a-cc55-4f33-a7f4-257872b8da05","Type":"ContainerStarted","Data":"819f27f7f530e06574f990fae511658a82da95b95f3a3d7abf31f85b312bb572"}
Jul 06 18:26:09.313122 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:09.313085    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ccj5r" event={"ID":"177ab09a-cc55-4f33-a7f4-257872b8da05","Type":"ContainerStarted","Data":"015f379447da842a95bbf7197911414476a4b6965878b503b5e153b9c97cd138"}
Jul 06 18:26:09.313550 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:09.313163    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ccj5r"
Jul 06 18:26:09.314527 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:09.314507    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hdrzx" event={"ID":"5f34ac69-ec56-4482-a77a-d60caac43ac0","Type":"ContainerStarted","Data":"6e518277647f1863f0a0bea50a58b6f6fd059c9928412dae7c1759e0957b77ec"}
Jul 06 18:26:09.314602 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:09.314543    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hdrzx" event={"ID":"5f34ac69-ec56-4482-a77a-d60caac43ac0","Type":"ContainerStarted","Data":"782d420d434c9b8c94d88284e6e5fb921c6de6111dadf09f69acb2d9f34df463"}
Jul 06 18:26:09.314602 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:09.314590    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hdrzx"
Jul 06 18:26:09.328164 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:09.328135    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="169ad069-4a3e-4909-800a-86927ad21092" path="/var/lib/kubelet/pods/169ad069-4a3e-4909-800a-86927ad21092/volumes"
Jul 06 18:26:09.328591 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:09.328571    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9b5cebf5-1671-4635-ab43-31f760286b43" path="/var/lib/kubelet/pods/9b5cebf5-1671-4635-ab43-31f760286b43/volumes"
Jul 06 18:26:09.335151 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:09.335109    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ccj5r" podStartSLOduration=2.335098091 podStartE2EDuration="2.335098091s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:26:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:26:09.334321796 +0000 UTC m=+504.491720282" watchObservedRunningTime="2026-07-06 18:26:09.335098091 +0000 UTC m=+504.492496577"
Jul 06 18:26:09.336520 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:26:09.336491    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-m4rgl\" is forbidden: User \"system:node:ip-10-0-134-113.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-113.ec2.internal' and this object" podUID="169ad069-4a3e-4909-800a-86927ad21092" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m4rgl"
Jul 06 18:26:09.339353 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:26:09.339329    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-hpv8d\" is forbidden: User \"system:node:ip-10-0-134-113.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-113.ec2.internal' and this object" podUID="9b5cebf5-1671-4635-ab43-31f760286b43" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hpv8d"
Jul 06 18:26:09.363419 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:09.363373    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hdrzx" podStartSLOduration=2.3633604630000002 podStartE2EDuration="2.363360463s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:26:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:26:09.362437204 +0000 UTC m=+504.519835687" watchObservedRunningTime="2026-07-06 18:26:09.363360463 +0000 UTC m=+504.520758949"
Jul 06 18:26:14.289846 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:14.289813    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-l47mq"
Jul 06 18:26:20.327636 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:20.327600    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hdrzx"
Jul 06 18:26:20.327999 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:20.327669    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ccj5r"
Jul 06 18:26:38.981874 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:38.981791    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ccj5r"]
Jul 06 18:26:38.982380 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:38.981985    2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ccj5r" podUID="177ab09a-cc55-4f33-a7f4-257872b8da05" containerName="manager" containerID="cri-o://819f27f7f530e06574f990fae511658a82da95b95f3a3d7abf31f85b312bb572" gracePeriod=10
Jul 06 18:26:39.219135 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:39.219111    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ccj5r"
Jul 06 18:26:39.234553 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:39.234493    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/177ab09a-cc55-4f33-a7f4-257872b8da05-extensions-socket-volume\") pod \"177ab09a-cc55-4f33-a7f4-257872b8da05\" (UID: \"177ab09a-cc55-4f33-a7f4-257872b8da05\") "
Jul 06 18:26:39.234553 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:39.234544    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t2cl9\" (UniqueName: \"kubernetes.io/projected/177ab09a-cc55-4f33-a7f4-257872b8da05-kube-api-access-t2cl9\") pod \"177ab09a-cc55-4f33-a7f4-257872b8da05\" (UID: \"177ab09a-cc55-4f33-a7f4-257872b8da05\") "
Jul 06 18:26:39.234937 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:39.234911    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/177ab09a-cc55-4f33-a7f4-257872b8da05-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "177ab09a-cc55-4f33-a7f4-257872b8da05" (UID: "177ab09a-cc55-4f33-a7f4-257872b8da05"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 18:26:39.236543 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:39.236523    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/177ab09a-cc55-4f33-a7f4-257872b8da05-kube-api-access-t2cl9" (OuterVolumeSpecName: "kube-api-access-t2cl9") pod "177ab09a-cc55-4f33-a7f4-257872b8da05" (UID: "177ab09a-cc55-4f33-a7f4-257872b8da05"). InnerVolumeSpecName "kube-api-access-t2cl9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 18:26:39.335255 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:39.335232    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t2cl9\" (UniqueName: \"kubernetes.io/projected/177ab09a-cc55-4f33-a7f4-257872b8da05-kube-api-access-t2cl9\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:26:39.335386 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:39.335257    2580 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/177ab09a-cc55-4f33-a7f4-257872b8da05-extensions-socket-volume\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:26:39.442558 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:39.442520    2580 generic.go:356] "Generic (PLEG): container finished" podID="177ab09a-cc55-4f33-a7f4-257872b8da05" containerID="819f27f7f530e06574f990fae511658a82da95b95f3a3d7abf31f85b312bb572" exitCode=0
Jul 06 18:26:39.442762 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:39.442573    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ccj5r"
Jul 06 18:26:39.442762 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:39.442578    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ccj5r" event={"ID":"177ab09a-cc55-4f33-a7f4-257872b8da05","Type":"ContainerDied","Data":"819f27f7f530e06574f990fae511658a82da95b95f3a3d7abf31f85b312bb572"}
Jul 06 18:26:39.442762 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:39.442633    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ccj5r" event={"ID":"177ab09a-cc55-4f33-a7f4-257872b8da05","Type":"ContainerDied","Data":"015f379447da842a95bbf7197911414476a4b6965878b503b5e153b9c97cd138"}
Jul 06 18:26:39.442762 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:39.442661    2580 scope.go:117] "RemoveContainer" containerID="819f27f7f530e06574f990fae511658a82da95b95f3a3d7abf31f85b312bb572"
Jul 06 18:26:39.451085 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:39.451064    2580 scope.go:117] "RemoveContainer" containerID="819f27f7f530e06574f990fae511658a82da95b95f3a3d7abf31f85b312bb572"
Jul 06 18:26:39.451346 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:26:39.451328    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"819f27f7f530e06574f990fae511658a82da95b95f3a3d7abf31f85b312bb572\": container with ID starting with 819f27f7f530e06574f990fae511658a82da95b95f3a3d7abf31f85b312bb572 not found: ID does not exist" containerID="819f27f7f530e06574f990fae511658a82da95b95f3a3d7abf31f85b312bb572"
Jul 06 18:26:39.451393 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:39.451355    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"819f27f7f530e06574f990fae511658a82da95b95f3a3d7abf31f85b312bb572"} err="failed to get container status \"819f27f7f530e06574f990fae511658a82da95b95f3a3d7abf31f85b312bb572\": rpc error: code = NotFound desc = could not find container \"819f27f7f530e06574f990fae511658a82da95b95f3a3d7abf31f85b312bb572\": container with ID starting with 819f27f7f530e06574f990fae511658a82da95b95f3a3d7abf31f85b312bb572 not found: ID does not exist"
Jul 06 18:26:39.462350 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:39.462327    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ccj5r"]
Jul 06 18:26:39.466209 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:39.466188    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-ccj5r"]
Jul 06 18:26:41.322367 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:41.322327    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="177ab09a-cc55-4f33-a7f4-257872b8da05" path="/var/lib/kubelet/pods/177ab09a-cc55-4f33-a7f4-257872b8da05/volumes"
Jul 06 18:26:55.449560 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:55.449522    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tlmbg"]
Jul 06 18:26:55.450167 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:55.449991    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="177ab09a-cc55-4f33-a7f4-257872b8da05" containerName="manager"
Jul 06 18:26:55.450167 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:55.450010    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="177ab09a-cc55-4f33-a7f4-257872b8da05" containerName="manager"
Jul 06 18:26:55.450167 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:55.450120    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="177ab09a-cc55-4f33-a7f4-257872b8da05" containerName="manager"
Jul 06 18:26:55.452798 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:55.452777    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tlmbg"
Jul 06 18:26:55.456049 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:55.456028    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-d9sfb\""
Jul 06 18:26:55.465206 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:55.465181    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tlmbg"]
Jul 06 18:26:55.558033 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:55.557977    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/75b7f81b-fdb8-4d09-8317-8c9323960ed5-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-tlmbg\" (UID: \"75b7f81b-fdb8-4d09-8317-8c9323960ed5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tlmbg"
Jul 06 18:26:55.558033 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:55.558036    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/75b7f81b-fdb8-4d09-8317-8c9323960ed5-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-tlmbg\" (UID: \"75b7f81b-fdb8-4d09-8317-8c9323960ed5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tlmbg"
Jul 06 18:26:55.558241 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:55.558074    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/75b7f81b-fdb8-4d09-8317-8c9323960ed5-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-tlmbg\" (UID: \"75b7f81b-fdb8-4d09-8317-8c9323960ed5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tlmbg"
Jul 06 18:26:55.558241 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:55.558097    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dggp5\" (UniqueName: \"kubernetes.io/projected/75b7f81b-fdb8-4d09-8317-8c9323960ed5-kube-api-access-dggp5\") pod \"maas-default-gateway-openshift-default-56654dcf7-tlmbg\" (UID: \"75b7f81b-fdb8-4d09-8317-8c9323960ed5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tlmbg"
Jul 06 18:26:55.558241 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:55.558130    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/75b7f81b-fdb8-4d09-8317-8c9323960ed5-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-tlmbg\" (UID: \"75b7f81b-fdb8-4d09-8317-8c9323960ed5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tlmbg"
Jul 06 18:26:55.558241 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:55.558161    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/75b7f81b-fdb8-4d09-8317-8c9323960ed5-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-tlmbg\" (UID: \"75b7f81b-fdb8-4d09-8317-8c9323960ed5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tlmbg"
Jul 06 18:26:55.558241 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:55.558192    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/75b7f81b-fdb8-4d09-8317-8c9323960ed5-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-tlmbg\" (UID: \"75b7f81b-fdb8-4d09-8317-8c9323960ed5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tlmbg"
Jul 06 18:26:55.558241 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:55.558212    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/75b7f81b-fdb8-4d09-8317-8c9323960ed5-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-tlmbg\" (UID: \"75b7f81b-fdb8-4d09-8317-8c9323960ed5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tlmbg"
Jul 06 18:26:55.558420 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:55.558266    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/75b7f81b-fdb8-4d09-8317-8c9323960ed5-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-tlmbg\" (UID: \"75b7f81b-fdb8-4d09-8317-8c9323960ed5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tlmbg"
Jul 06 18:26:55.659160 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:55.659120    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/75b7f81b-fdb8-4d09-8317-8c9323960ed5-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-tlmbg\" (UID: \"75b7f81b-fdb8-4d09-8317-8c9323960ed5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tlmbg"
Jul 06 18:26:55.659160 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:55.659160    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/75b7f81b-fdb8-4d09-8317-8c9323960ed5-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-tlmbg\" (UID: \"75b7f81b-fdb8-4d09-8317-8c9323960ed5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tlmbg"
Jul 06 18:26:55.659417 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:55.659183    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/75b7f81b-fdb8-4d09-8317-8c9323960ed5-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-tlmbg\" (UID: \"75b7f81b-fdb8-4d09-8317-8c9323960ed5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tlmbg"
Jul 06 18:26:55.659417 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:55.659211    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/75b7f81b-fdb8-4d09-8317-8c9323960ed5-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-tlmbg\" (UID: \"75b7f81b-fdb8-4d09-8317-8c9323960ed5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tlmbg"
Jul 06 18:26:55.659417 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:55.659229    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dggp5\" (UniqueName: \"kubernetes.io/projected/75b7f81b-fdb8-4d09-8317-8c9323960ed5-kube-api-access-dggp5\") pod \"maas-default-gateway-openshift-default-56654dcf7-tlmbg\" (UID: \"75b7f81b-fdb8-4d09-8317-8c9323960ed5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tlmbg"
Jul 06 18:26:55.659417 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:55.659256    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/75b7f81b-fdb8-4d09-8317-8c9323960ed5-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-tlmbg\" (UID: \"75b7f81b-fdb8-4d09-8317-8c9323960ed5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tlmbg"
Jul 06 18:26:55.659417 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:55.659276    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/75b7f81b-fdb8-4d09-8317-8c9323960ed5-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-tlmbg\" (UID: \"75b7f81b-fdb8-4d09-8317-8c9323960ed5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tlmbg"
Jul 06 18:26:55.659417 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:55.659311    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/75b7f81b-fdb8-4d09-8317-8c9323960ed5-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-tlmbg\" (UID: \"75b7f81b-fdb8-4d09-8317-8c9323960ed5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tlmbg"
Jul 06 18:26:55.659417 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:55.659340    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/75b7f81b-fdb8-4d09-8317-8c9323960ed5-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-tlmbg\" (UID: \"75b7f81b-fdb8-4d09-8317-8c9323960ed5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tlmbg"
Jul 06 18:26:55.659726 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:55.659597    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/75b7f81b-fdb8-4d09-8317-8c9323960ed5-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-tlmbg\" (UID: \"75b7f81b-fdb8-4d09-8317-8c9323960ed5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tlmbg"
Jul 06 18:26:55.659772 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:55.659724    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/75b7f81b-fdb8-4d09-8317-8c9323960ed5-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-tlmbg\" (UID: \"75b7f81b-fdb8-4d09-8317-8c9323960ed5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tlmbg"
Jul 06 18:26:55.659772 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:55.659755    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/75b7f81b-fdb8-4d09-8317-8c9323960ed5-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-tlmbg\" (UID: \"75b7f81b-fdb8-4d09-8317-8c9323960ed5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tlmbg"
Jul 06 18:26:55.659904 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:55.659883    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/75b7f81b-fdb8-4d09-8317-8c9323960ed5-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-tlmbg\" (UID: \"75b7f81b-fdb8-4d09-8317-8c9323960ed5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tlmbg"
Jul 06 18:26:55.659976 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:55.659958    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/75b7f81b-fdb8-4d09-8317-8c9323960ed5-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-tlmbg\" (UID: \"75b7f81b-fdb8-4d09-8317-8c9323960ed5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tlmbg"
Jul 06 18:26:55.661665 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:55.661638    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/75b7f81b-fdb8-4d09-8317-8c9323960ed5-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-tlmbg\" (UID: \"75b7f81b-fdb8-4d09-8317-8c9323960ed5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tlmbg"
Jul 06 18:26:55.661868 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:55.661853    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/75b7f81b-fdb8-4d09-8317-8c9323960ed5-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-tlmbg\" (UID: \"75b7f81b-fdb8-4d09-8317-8c9323960ed5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tlmbg"
Jul 06 18:26:55.673853 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:55.673825    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dggp5\" (UniqueName: \"kubernetes.io/projected/75b7f81b-fdb8-4d09-8317-8c9323960ed5-kube-api-access-dggp5\") pod \"maas-default-gateway-openshift-default-56654dcf7-tlmbg\" (UID: \"75b7f81b-fdb8-4d09-8317-8c9323960ed5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tlmbg"
Jul 06 18:26:55.674317 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:55.674299    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/75b7f81b-fdb8-4d09-8317-8c9323960ed5-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-tlmbg\" (UID: \"75b7f81b-fdb8-4d09-8317-8c9323960ed5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tlmbg"
Jul 06 18:26:55.763479 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:55.763400    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tlmbg"
Jul 06 18:26:55.893450 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:55.893418    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tlmbg"]
Jul 06 18:26:55.896451 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:26:55.896427    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod75b7f81b_fdb8_4d09_8317_8c9323960ed5.slice/crio-a3c1a9581ac55d432488273c4f06c4d0e50d3aa70613f0ed9feb4043bc18f56e WatchSource:0}: Error finding container a3c1a9581ac55d432488273c4f06c4d0e50d3aa70613f0ed9feb4043bc18f56e: Status 404 returned error can't find the container with id a3c1a9581ac55d432488273c4f06c4d0e50d3aa70613f0ed9feb4043bc18f56e
Jul 06 18:26:55.898546 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:55.898515    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 06 18:26:55.898626 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:55.898579    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 06 18:26:55.898626 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:55.898607    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 06 18:26:56.518288 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:56.518252    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tlmbg" event={"ID":"75b7f81b-fdb8-4d09-8317-8c9323960ed5","Type":"ContainerStarted","Data":"6e422c408719f68333aba3d4026d60f0789c2bc4f794fd76de1ecce6c0b6f4e4"}
Jul 06 18:26:56.518762 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:56.518321    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tlmbg" event={"ID":"75b7f81b-fdb8-4d09-8317-8c9323960ed5","Type":"ContainerStarted","Data":"a3c1a9581ac55d432488273c4f06c4d0e50d3aa70613f0ed9feb4043bc18f56e"}
Jul 06 18:26:56.540152 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:56.540101    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tlmbg" podStartSLOduration=1.5400849970000001 podStartE2EDuration="1.540084997s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:26:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:26:56.539443364 +0000 UTC m=+551.696841864" watchObservedRunningTime="2026-07-06 18:26:56.540084997 +0000 UTC m=+551.697483542"
Jul 06 18:26:56.764378 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:56.764343    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tlmbg"
Jul 06 18:26:56.769302 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:56.769229    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tlmbg"
Jul 06 18:26:57.522435 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:57.522402    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tlmbg"
Jul 06 18:26:57.523516 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:57.523498    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tlmbg"
Jul 06 18:26:57.680412 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:57.680377    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-7wptq"]
Jul 06 18:26:57.682694 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:57.682678    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-7wptq"
Jul 06 18:26:57.687545 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:57.687524    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-5p2gs\""
Jul 06 18:26:57.687682 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:57.687567    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 06 18:26:57.695511 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:57.695488    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-7wptq"]
Jul 06 18:26:57.777070 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:57.776973    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9nnmh\" (UniqueName: \"kubernetes.io/projected/aff56d3e-8ebb-4a9e-b6fc-a044754d49a0-kube-api-access-9nnmh\") pod \"limitador-limitador-64c94fd64f-7wptq\" (UID: \"aff56d3e-8ebb-4a9e-b6fc-a044754d49a0\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-7wptq"
Jul 06 18:26:57.777244 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:57.777066    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/aff56d3e-8ebb-4a9e-b6fc-a044754d49a0-config-file\") pod \"limitador-limitador-64c94fd64f-7wptq\" (UID: \"aff56d3e-8ebb-4a9e-b6fc-a044754d49a0\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-7wptq"
Jul 06 18:26:57.778601 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:57.778580    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-7wptq"]
Jul 06 18:26:57.878081 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:57.878043    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9nnmh\" (UniqueName: \"kubernetes.io/projected/aff56d3e-8ebb-4a9e-b6fc-a044754d49a0-kube-api-access-9nnmh\") pod \"limitador-limitador-64c94fd64f-7wptq\" (UID: \"aff56d3e-8ebb-4a9e-b6fc-a044754d49a0\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-7wptq"
Jul 06 18:26:57.879089 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:57.878364    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/aff56d3e-8ebb-4a9e-b6fc-a044754d49a0-config-file\") pod \"limitador-limitador-64c94fd64f-7wptq\" (UID: \"aff56d3e-8ebb-4a9e-b6fc-a044754d49a0\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-7wptq"
Jul 06 18:26:57.879446 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:57.879421    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/aff56d3e-8ebb-4a9e-b6fc-a044754d49a0-config-file\") pod \"limitador-limitador-64c94fd64f-7wptq\" (UID: \"aff56d3e-8ebb-4a9e-b6fc-a044754d49a0\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-7wptq"
Jul 06 18:26:57.886268 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:57.886245    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9nnmh\" (UniqueName: \"kubernetes.io/projected/aff56d3e-8ebb-4a9e-b6fc-a044754d49a0-kube-api-access-9nnmh\") pod \"limitador-limitador-64c94fd64f-7wptq\" (UID: \"aff56d3e-8ebb-4a9e-b6fc-a044754d49a0\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-7wptq"
Jul 06 18:26:57.993373 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:57.993332    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-7wptq"
Jul 06 18:26:58.129367 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:58.129341    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-7wptq"]
Jul 06 18:26:58.131960 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:26:58.131933    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaff56d3e_8ebb_4a9e_b6fc_a044754d49a0.slice/crio-98d800901c67f4b8915fa532faef5e8a717a695d9fa14fd07851de6985b2da58 WatchSource:0}: Error finding container 98d800901c67f4b8915fa532faef5e8a717a695d9fa14fd07851de6985b2da58: Status 404 returned error can't find the container with id 98d800901c67f4b8915fa532faef5e8a717a695d9fa14fd07851de6985b2da58
Jul 06 18:26:58.469849 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:58.469817    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-772rr"]
Jul 06 18:26:58.472687 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:58.472670    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-772rr"
Jul 06 18:26:58.475587 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:58.475568    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-8zw9t\""
Jul 06 18:26:58.480316 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:58.480269    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-772rr"]
Jul 06 18:26:58.527771 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:58.527735    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-7wptq" event={"ID":"aff56d3e-8ebb-4a9e-b6fc-a044754d49a0","Type":"ContainerStarted","Data":"98d800901c67f4b8915fa532faef5e8a717a695d9fa14fd07851de6985b2da58"}
Jul 06 18:26:58.585855 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:58.585822    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dd7wc\" (UniqueName: \"kubernetes.io/projected/b968cbd0-0023-4830-8025-6fe32b2411b6-kube-api-access-dd7wc\") pod \"authorino-58cd955675-772rr\" (UID: \"b968cbd0-0023-4830-8025-6fe32b2411b6\") " pod="kuadrant-system/authorino-58cd955675-772rr"
Jul 06 18:26:58.686975 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:58.686939    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dd7wc\" (UniqueName: \"kubernetes.io/projected/b968cbd0-0023-4830-8025-6fe32b2411b6-kube-api-access-dd7wc\") pod \"authorino-58cd955675-772rr\" (UID: \"b968cbd0-0023-4830-8025-6fe32b2411b6\") " pod="kuadrant-system/authorino-58cd955675-772rr"
Jul 06 18:26:58.697503 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:58.697479    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dd7wc\" (UniqueName: \"kubernetes.io/projected/b968cbd0-0023-4830-8025-6fe32b2411b6-kube-api-access-dd7wc\") pod \"authorino-58cd955675-772rr\" (UID: \"b968cbd0-0023-4830-8025-6fe32b2411b6\") " pod="kuadrant-system/authorino-58cd955675-772rr"
Jul 06 18:26:58.783297 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:58.783229    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-772rr"
Jul 06 18:26:58.907322 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:58.907291    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-772rr"]
Jul 06 18:26:58.910910 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:26:58.910880    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb968cbd0_0023_4830_8025_6fe32b2411b6.slice/crio-0f7d3ce65c53a4afd174a41a61d872e86c3d1fc4a9a5f42a0b5b9a4734538fbb WatchSource:0}: Error finding container 0f7d3ce65c53a4afd174a41a61d872e86c3d1fc4a9a5f42a0b5b9a4734538fbb: Status 404 returned error can't find the container with id 0f7d3ce65c53a4afd174a41a61d872e86c3d1fc4a9a5f42a0b5b9a4734538fbb
Jul 06 18:26:59.535421 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:26:59.535373    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-772rr" event={"ID":"b968cbd0-0023-4830-8025-6fe32b2411b6","Type":"ContainerStarted","Data":"0f7d3ce65c53a4afd174a41a61d872e86c3d1fc4a9a5f42a0b5b9a4734538fbb"}
Jul 06 18:27:01.551702 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:01.551664    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-7wptq" event={"ID":"aff56d3e-8ebb-4a9e-b6fc-a044754d49a0","Type":"ContainerStarted","Data":"5b4377a641c169b4983be4ae18cdded010806997613d23cc9a6abd2498c5a15c"}
Jul 06 18:27:01.552144 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:01.551845    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-7wptq"
Jul 06 18:27:01.569908 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:01.569849    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-7wptq" podStartSLOduration=1.636113183 podStartE2EDuration="4.56983034s" totalImagesPullingTime="2.933717157s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:26:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:27:01.567955113 +0000 UTC m=+556.725353599" watchObservedRunningTime="2026-07-06 18:27:01.56983034 +0000 UTC m=+556.727228872"
Jul 06 18:27:03.157435 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:03.157353    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-772rr"]
Jul 06 18:27:03.564788 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:03.564700    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-772rr" event={"ID":"b968cbd0-0023-4830-8025-6fe32b2411b6","Type":"ContainerStarted","Data":"81a8546b6dff016b340c8add6738bbbd3ccd39aafa7ecb3fef31cc9a39207311"}
Jul 06 18:27:03.580074 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:03.579993    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-772rr" podStartSLOduration=1.6288970809999999 podStartE2EDuration="5.579976724s" totalImagesPullingTime="3.951079643s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:26:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:27:03.579244593 +0000 UTC m=+558.736643080" watchObservedRunningTime="2026-07-06 18:27:03.579976724 +0000 UTC m=+558.737375210"
Jul 06 18:27:04.569412 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:04.569366    2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-772rr" podUID="b968cbd0-0023-4830-8025-6fe32b2411b6" containerName="authorino" containerID="cri-o://81a8546b6dff016b340c8add6738bbbd3ccd39aafa7ecb3fef31cc9a39207311" gracePeriod=30
Jul 06 18:27:04.824428 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:04.824409    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-772rr"
Jul 06 18:27:04.946470 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:04.946436    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dd7wc\" (UniqueName: \"kubernetes.io/projected/b968cbd0-0023-4830-8025-6fe32b2411b6-kube-api-access-dd7wc\") pod \"b968cbd0-0023-4830-8025-6fe32b2411b6\" (UID: \"b968cbd0-0023-4830-8025-6fe32b2411b6\") "
Jul 06 18:27:04.948954 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:04.948920    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b968cbd0-0023-4830-8025-6fe32b2411b6-kube-api-access-dd7wc" (OuterVolumeSpecName: "kube-api-access-dd7wc") pod "b968cbd0-0023-4830-8025-6fe32b2411b6" (UID: "b968cbd0-0023-4830-8025-6fe32b2411b6"). InnerVolumeSpecName "kube-api-access-dd7wc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 18:27:05.047476 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:05.047440    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dd7wc\" (UniqueName: \"kubernetes.io/projected/b968cbd0-0023-4830-8025-6fe32b2411b6-kube-api-access-dd7wc\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:27:05.574708 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:05.574672    2580 generic.go:356] "Generic (PLEG): container finished" podID="b968cbd0-0023-4830-8025-6fe32b2411b6" containerID="81a8546b6dff016b340c8add6738bbbd3ccd39aafa7ecb3fef31cc9a39207311" exitCode=0
Jul 06 18:27:05.575213 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:05.574713    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-772rr"
Jul 06 18:27:05.575213 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:05.574721    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-772rr" event={"ID":"b968cbd0-0023-4830-8025-6fe32b2411b6","Type":"ContainerDied","Data":"81a8546b6dff016b340c8add6738bbbd3ccd39aafa7ecb3fef31cc9a39207311"}
Jul 06 18:27:05.575213 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:05.574760    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-772rr" event={"ID":"b968cbd0-0023-4830-8025-6fe32b2411b6","Type":"ContainerDied","Data":"0f7d3ce65c53a4afd174a41a61d872e86c3d1fc4a9a5f42a0b5b9a4734538fbb"}
Jul 06 18:27:05.575213 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:05.574781    2580 scope.go:117] "RemoveContainer" containerID="81a8546b6dff016b340c8add6738bbbd3ccd39aafa7ecb3fef31cc9a39207311"
Jul 06 18:27:05.582854 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:05.582839    2580 scope.go:117] "RemoveContainer" containerID="81a8546b6dff016b340c8add6738bbbd3ccd39aafa7ecb3fef31cc9a39207311"
Jul 06 18:27:05.583151 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:27:05.583131    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"81a8546b6dff016b340c8add6738bbbd3ccd39aafa7ecb3fef31cc9a39207311\": container with ID starting with 81a8546b6dff016b340c8add6738bbbd3ccd39aafa7ecb3fef31cc9a39207311 not found: ID does not exist" containerID="81a8546b6dff016b340c8add6738bbbd3ccd39aafa7ecb3fef31cc9a39207311"
Jul 06 18:27:05.583204 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:05.583165    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"81a8546b6dff016b340c8add6738bbbd3ccd39aafa7ecb3fef31cc9a39207311"} err="failed to get container status \"81a8546b6dff016b340c8add6738bbbd3ccd39aafa7ecb3fef31cc9a39207311\": rpc error: code = NotFound desc = could not find container \"81a8546b6dff016b340c8add6738bbbd3ccd39aafa7ecb3fef31cc9a39207311\": container with ID starting with 81a8546b6dff016b340c8add6738bbbd3ccd39aafa7ecb3fef31cc9a39207311 not found: ID does not exist"
Jul 06 18:27:05.591280 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:05.591257    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-772rr"]
Jul 06 18:27:05.595261 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:05.595243    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-772rr"]
Jul 06 18:27:07.328076 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:07.328042    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b968cbd0-0023-4830-8025-6fe32b2411b6" path="/var/lib/kubelet/pods/b968cbd0-0023-4830-8025-6fe32b2411b6/volumes"
Jul 06 18:27:12.558129 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:12.558098    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-7wptq"
Jul 06 18:27:12.622960 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:12.622926    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-7wptq"]
Jul 06 18:27:12.623168 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:12.623145    2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-7wptq" podUID="aff56d3e-8ebb-4a9e-b6fc-a044754d49a0" containerName="limitador" containerID="cri-o://5b4377a641c169b4983be4ae18cdded010806997613d23cc9a6abd2498c5a15c" gracePeriod=30
Jul 06 18:27:13.179156 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:13.179133    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-7wptq"
Jul 06 18:27:13.317929 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:13.317834    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/aff56d3e-8ebb-4a9e-b6fc-a044754d49a0-config-file\") pod \"aff56d3e-8ebb-4a9e-b6fc-a044754d49a0\" (UID: \"aff56d3e-8ebb-4a9e-b6fc-a044754d49a0\") "
Jul 06 18:27:13.317929 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:13.317900    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9nnmh\" (UniqueName: \"kubernetes.io/projected/aff56d3e-8ebb-4a9e-b6fc-a044754d49a0-kube-api-access-9nnmh\") pod \"aff56d3e-8ebb-4a9e-b6fc-a044754d49a0\" (UID: \"aff56d3e-8ebb-4a9e-b6fc-a044754d49a0\") "
Jul 06 18:27:13.318283 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:13.318258    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aff56d3e-8ebb-4a9e-b6fc-a044754d49a0-config-file" (OuterVolumeSpecName: "config-file") pod "aff56d3e-8ebb-4a9e-b6fc-a044754d49a0" (UID: "aff56d3e-8ebb-4a9e-b6fc-a044754d49a0"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 18:27:13.319938 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:13.319914    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aff56d3e-8ebb-4a9e-b6fc-a044754d49a0-kube-api-access-9nnmh" (OuterVolumeSpecName: "kube-api-access-9nnmh") pod "aff56d3e-8ebb-4a9e-b6fc-a044754d49a0" (UID: "aff56d3e-8ebb-4a9e-b6fc-a044754d49a0"). InnerVolumeSpecName "kube-api-access-9nnmh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 18:27:13.418590 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:13.418549    2580 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/aff56d3e-8ebb-4a9e-b6fc-a044754d49a0-config-file\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:27:13.418590 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:13.418589    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9nnmh\" (UniqueName: \"kubernetes.io/projected/aff56d3e-8ebb-4a9e-b6fc-a044754d49a0-kube-api-access-9nnmh\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:27:13.609691 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:13.609656    2580 generic.go:356] "Generic (PLEG): container finished" podID="aff56d3e-8ebb-4a9e-b6fc-a044754d49a0" containerID="5b4377a641c169b4983be4ae18cdded010806997613d23cc9a6abd2498c5a15c" exitCode=0
Jul 06 18:27:13.610185 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:13.609721    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-7wptq"
Jul 06 18:27:13.610185 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:13.609747    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-7wptq" event={"ID":"aff56d3e-8ebb-4a9e-b6fc-a044754d49a0","Type":"ContainerDied","Data":"5b4377a641c169b4983be4ae18cdded010806997613d23cc9a6abd2498c5a15c"}
Jul 06 18:27:13.610185 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:13.609833    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-7wptq" event={"ID":"aff56d3e-8ebb-4a9e-b6fc-a044754d49a0","Type":"ContainerDied","Data":"98d800901c67f4b8915fa532faef5e8a717a695d9fa14fd07851de6985b2da58"}
Jul 06 18:27:13.610185 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:13.609856    2580 scope.go:117] "RemoveContainer" containerID="5b4377a641c169b4983be4ae18cdded010806997613d23cc9a6abd2498c5a15c"
Jul 06 18:27:13.617968 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:13.617950    2580 scope.go:117] "RemoveContainer" containerID="5b4377a641c169b4983be4ae18cdded010806997613d23cc9a6abd2498c5a15c"
Jul 06 18:27:13.618237 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:27:13.618216    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5b4377a641c169b4983be4ae18cdded010806997613d23cc9a6abd2498c5a15c\": container with ID starting with 5b4377a641c169b4983be4ae18cdded010806997613d23cc9a6abd2498c5a15c not found: ID does not exist" containerID="5b4377a641c169b4983be4ae18cdded010806997613d23cc9a6abd2498c5a15c"
Jul 06 18:27:13.618281 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:13.618245    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5b4377a641c169b4983be4ae18cdded010806997613d23cc9a6abd2498c5a15c"} err="failed to get container status \"5b4377a641c169b4983be4ae18cdded010806997613d23cc9a6abd2498c5a15c\": rpc error: code = NotFound desc = could not find container \"5b4377a641c169b4983be4ae18cdded010806997613d23cc9a6abd2498c5a15c\": container with ID starting with 5b4377a641c169b4983be4ae18cdded010806997613d23cc9a6abd2498c5a15c not found: ID does not exist"
Jul 06 18:27:13.626369 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:13.626345    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-7wptq"]
Jul 06 18:27:13.630028 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:13.629996    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-7wptq"]
Jul 06 18:27:14.147915 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:14.147881    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/postgres-69fb4594cf-859xw"]
Jul 06 18:27:14.148263 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:14.148240    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aff56d3e-8ebb-4a9e-b6fc-a044754d49a0" containerName="limitador"
Jul 06 18:27:14.148263 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:14.148266    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="aff56d3e-8ebb-4a9e-b6fc-a044754d49a0" containerName="limitador"
Jul 06 18:27:14.148361 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:14.148275    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b968cbd0-0023-4830-8025-6fe32b2411b6" containerName="authorino"
Jul 06 18:27:14.148361 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:14.148281    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="b968cbd0-0023-4830-8025-6fe32b2411b6" containerName="authorino"
Jul 06 18:27:14.148361 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:14.148350    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="aff56d3e-8ebb-4a9e-b6fc-a044754d49a0" containerName="limitador"
Jul 06 18:27:14.148361 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:14.148360    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="b968cbd0-0023-4830-8025-6fe32b2411b6" containerName="authorino"
Jul 06 18:27:14.152501 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:14.152479    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-69fb4594cf-859xw"
Jul 06 18:27:14.155245 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:14.155223    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-g5nz7\""
Jul 06 18:27:14.155359 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:14.155224    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"postgres-creds\""
Jul 06 18:27:14.158551 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:14.158526    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-69fb4594cf-859xw"]
Jul 06 18:27:14.225041 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:14.224987    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ql4z8\" (UniqueName: \"kubernetes.io/projected/c481500b-94ce-4b34-8a62-755f86931529-kube-api-access-ql4z8\") pod \"postgres-69fb4594cf-859xw\" (UID: \"c481500b-94ce-4b34-8a62-755f86931529\") " pod="opendatahub/postgres-69fb4594cf-859xw"
Jul 06 18:27:14.225213 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:14.225062    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/c481500b-94ce-4b34-8a62-755f86931529-data\") pod \"postgres-69fb4594cf-859xw\" (UID: \"c481500b-94ce-4b34-8a62-755f86931529\") " pod="opendatahub/postgres-69fb4594cf-859xw"
Jul 06 18:27:14.326003 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:14.325970    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ql4z8\" (UniqueName: \"kubernetes.io/projected/c481500b-94ce-4b34-8a62-755f86931529-kube-api-access-ql4z8\") pod \"postgres-69fb4594cf-859xw\" (UID: \"c481500b-94ce-4b34-8a62-755f86931529\") " pod="opendatahub/postgres-69fb4594cf-859xw"
Jul 06 18:27:14.326218 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:14.326071    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/c481500b-94ce-4b34-8a62-755f86931529-data\") pod \"postgres-69fb4594cf-859xw\" (UID: \"c481500b-94ce-4b34-8a62-755f86931529\") " pod="opendatahub/postgres-69fb4594cf-859xw"
Jul 06 18:27:14.326345 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:14.326328    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/c481500b-94ce-4b34-8a62-755f86931529-data\") pod \"postgres-69fb4594cf-859xw\" (UID: \"c481500b-94ce-4b34-8a62-755f86931529\") " pod="opendatahub/postgres-69fb4594cf-859xw"
Jul 06 18:27:14.334525 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:14.334503    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ql4z8\" (UniqueName: \"kubernetes.io/projected/c481500b-94ce-4b34-8a62-755f86931529-kube-api-access-ql4z8\") pod \"postgres-69fb4594cf-859xw\" (UID: \"c481500b-94ce-4b34-8a62-755f86931529\") " pod="opendatahub/postgres-69fb4594cf-859xw"
Jul 06 18:27:14.464898 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:14.464795    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-69fb4594cf-859xw"
Jul 06 18:27:14.606079 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:14.606054    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-69fb4594cf-859xw"]
Jul 06 18:27:14.607992 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:27:14.607959    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc481500b_94ce_4b34_8a62_755f86931529.slice/crio-7d68ed86aed66f2a60d689b54baf77ff937c8bf790146760bc327984e6738d73 WatchSource:0}: Error finding container 7d68ed86aed66f2a60d689b54baf77ff937c8bf790146760bc327984e6738d73: Status 404 returned error can't find the container with id 7d68ed86aed66f2a60d689b54baf77ff937c8bf790146760bc327984e6738d73
Jul 06 18:27:14.617471 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:14.617441    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-69fb4594cf-859xw" event={"ID":"c481500b-94ce-4b34-8a62-755f86931529","Type":"ContainerStarted","Data":"7d68ed86aed66f2a60d689b54baf77ff937c8bf790146760bc327984e6738d73"}
Jul 06 18:27:15.327661 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:15.327621    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aff56d3e-8ebb-4a9e-b6fc-a044754d49a0" path="/var/lib/kubelet/pods/aff56d3e-8ebb-4a9e-b6fc-a044754d49a0/volumes"
Jul 06 18:27:19.649816 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:19.649716    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-69fb4594cf-859xw" event={"ID":"c481500b-94ce-4b34-8a62-755f86931529","Type":"ContainerStarted","Data":"8ca8870ca0a05cd22cc7c6da14f309138f974df3e455b214cfc293d1edfb8955"}
Jul 06 18:27:19.649816 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:19.649795    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/postgres-69fb4594cf-859xw"
Jul 06 18:27:19.666953 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:19.666724    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/postgres-69fb4594cf-859xw" podStartSLOduration=0.935357634 podStartE2EDuration="5.66670827s" totalImagesPullingTime="4.731350636s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:27:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:27:19.666264542 +0000 UTC m=+574.823663029" watchObservedRunningTime="2026-07-06 18:27:19.66670827 +0000 UTC m=+574.824106756"
Jul 06 18:27:25.685133 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:25.685101    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/postgres-69fb4594cf-859xw"
Jul 06 18:27:26.227692 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:26.227651    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-8stt9"]
Jul 06 18:27:26.233878 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:26.233853    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-8stt9"
Jul 06 18:27:26.236967 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:26.236939    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-8zw9t\""
Jul 06 18:27:26.238610 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:26.238572    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ps64q\" (UniqueName: \"kubernetes.io/projected/09d090f5-aea9-4603-b50e-91e490684569-kube-api-access-ps64q\") pod \"authorino-5d4b67c446-8stt9\" (UID: \"09d090f5-aea9-4603-b50e-91e490684569\") " pod="kuadrant-system/authorino-5d4b67c446-8stt9"
Jul 06 18:27:26.238776 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:26.238754    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-8stt9"]
Jul 06 18:27:26.339598 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:26.339561    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ps64q\" (UniqueName: \"kubernetes.io/projected/09d090f5-aea9-4603-b50e-91e490684569-kube-api-access-ps64q\") pod \"authorino-5d4b67c446-8stt9\" (UID: \"09d090f5-aea9-4603-b50e-91e490684569\") " pod="kuadrant-system/authorino-5d4b67c446-8stt9"
Jul 06 18:27:26.348139 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:26.348110    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ps64q\" (UniqueName: \"kubernetes.io/projected/09d090f5-aea9-4603-b50e-91e490684569-kube-api-access-ps64q\") pod \"authorino-5d4b67c446-8stt9\" (UID: \"09d090f5-aea9-4603-b50e-91e490684569\") " pod="kuadrant-system/authorino-5d4b67c446-8stt9"
Jul 06 18:27:26.479443 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:26.479364    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-8stt9"]
Jul 06 18:27:26.479659 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:26.479607    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-8stt9"
Jul 06 18:27:26.506236 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:26.506206    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6d7fb587f8-kdcpk"]
Jul 06 18:27:26.510469 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:26.510449    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d7fb587f8-kdcpk"
Jul 06 18:27:26.519783 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:26.519753    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6d7fb587f8-kdcpk"]
Jul 06 18:27:26.541370 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:26.541335    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z9bvc\" (UniqueName: \"kubernetes.io/projected/9a79f49f-e2d2-4873-9316-91e71019f330-kube-api-access-z9bvc\") pod \"authorino-6d7fb587f8-kdcpk\" (UID: \"9a79f49f-e2d2-4873-9316-91e71019f330\") " pod="kuadrant-system/authorino-6d7fb587f8-kdcpk"
Jul 06 18:27:26.619914 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:26.619866    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-8stt9"]
Jul 06 18:27:26.622219 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:27:26.622185    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod09d090f5_aea9_4603_b50e_91e490684569.slice/crio-02fbc2efa5bf23db83fb92d8301e8588989eb47db86c1bce2209ed1edc8723ff WatchSource:0}: Error finding container 02fbc2efa5bf23db83fb92d8301e8588989eb47db86c1bce2209ed1edc8723ff: Status 404 returned error can't find the container with id 02fbc2efa5bf23db83fb92d8301e8588989eb47db86c1bce2209ed1edc8723ff
Jul 06 18:27:26.641817 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:26.641785    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z9bvc\" (UniqueName: \"kubernetes.io/projected/9a79f49f-e2d2-4873-9316-91e71019f330-kube-api-access-z9bvc\") pod \"authorino-6d7fb587f8-kdcpk\" (UID: \"9a79f49f-e2d2-4873-9316-91e71019f330\") " pod="kuadrant-system/authorino-6d7fb587f8-kdcpk"
Jul 06 18:27:26.650144 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:26.650118    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z9bvc\" (UniqueName: \"kubernetes.io/projected/9a79f49f-e2d2-4873-9316-91e71019f330-kube-api-access-z9bvc\") pod \"authorino-6d7fb587f8-kdcpk\" (UID: \"9a79f49f-e2d2-4873-9316-91e71019f330\") " pod="kuadrant-system/authorino-6d7fb587f8-kdcpk"
Jul 06 18:27:26.680589 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:26.680557    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-8stt9" event={"ID":"09d090f5-aea9-4603-b50e-91e490684569","Type":"ContainerStarted","Data":"02fbc2efa5bf23db83fb92d8301e8588989eb47db86c1bce2209ed1edc8723ff"}
Jul 06 18:27:26.744468 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:26.744377    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6d7fb587f8-kdcpk"]
Jul 06 18:27:26.744822 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:26.744604    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d7fb587f8-kdcpk"
Jul 06 18:27:26.771130 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:26.771098    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-68d6459fbb-r2mqj"]
Jul 06 18:27:26.775439 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:26.775419    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-68d6459fbb-r2mqj"
Jul 06 18:27:26.778128 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:26.778109    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Jul 06 18:27:26.785544 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:26.785286    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-68d6459fbb-r2mqj"]
Jul 06 18:27:26.843546 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:26.843514    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xf77p\" (UniqueName: \"kubernetes.io/projected/ddfe9a5b-4ef3-4bad-8c95-9ab633bf51a2-kube-api-access-xf77p\") pod \"authorino-68d6459fbb-r2mqj\" (UID: \"ddfe9a5b-4ef3-4bad-8c95-9ab633bf51a2\") " pod="kuadrant-system/authorino-68d6459fbb-r2mqj"
Jul 06 18:27:26.843718 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:26.843672    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ddfe9a5b-4ef3-4bad-8c95-9ab633bf51a2-tls-cert\") pod \"authorino-68d6459fbb-r2mqj\" (UID: \"ddfe9a5b-4ef3-4bad-8c95-9ab633bf51a2\") " pod="kuadrant-system/authorino-68d6459fbb-r2mqj"
Jul 06 18:27:26.876049 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:26.876011    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6d7fb587f8-kdcpk"]
Jul 06 18:27:26.878259 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:27:26.878234    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9a79f49f_e2d2_4873_9316_91e71019f330.slice/crio-3b6c0e35ff065f07b5848376c387c811b31bf11fe9e9039c9188edd1cfda26b3 WatchSource:0}: Error finding container 3b6c0e35ff065f07b5848376c387c811b31bf11fe9e9039c9188edd1cfda26b3: Status 404 returned error can't find the container with id 3b6c0e35ff065f07b5848376c387c811b31bf11fe9e9039c9188edd1cfda26b3
Jul 06 18:27:26.944488 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:26.944443    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ddfe9a5b-4ef3-4bad-8c95-9ab633bf51a2-tls-cert\") pod \"authorino-68d6459fbb-r2mqj\" (UID: \"ddfe9a5b-4ef3-4bad-8c95-9ab633bf51a2\") " pod="kuadrant-system/authorino-68d6459fbb-r2mqj"
Jul 06 18:27:26.944488 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:26.944486    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xf77p\" (UniqueName: \"kubernetes.io/projected/ddfe9a5b-4ef3-4bad-8c95-9ab633bf51a2-kube-api-access-xf77p\") pod \"authorino-68d6459fbb-r2mqj\" (UID: \"ddfe9a5b-4ef3-4bad-8c95-9ab633bf51a2\") " pod="kuadrant-system/authorino-68d6459fbb-r2mqj"
Jul 06 18:27:26.946821 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:26.946796    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ddfe9a5b-4ef3-4bad-8c95-9ab633bf51a2-tls-cert\") pod \"authorino-68d6459fbb-r2mqj\" (UID: \"ddfe9a5b-4ef3-4bad-8c95-9ab633bf51a2\") " pod="kuadrant-system/authorino-68d6459fbb-r2mqj"
Jul 06 18:27:26.953257 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:26.953228    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xf77p\" (UniqueName: \"kubernetes.io/projected/ddfe9a5b-4ef3-4bad-8c95-9ab633bf51a2-kube-api-access-xf77p\") pod \"authorino-68d6459fbb-r2mqj\" (UID: \"ddfe9a5b-4ef3-4bad-8c95-9ab633bf51a2\") " pod="kuadrant-system/authorino-68d6459fbb-r2mqj"
Jul 06 18:27:27.086402 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:27.086368    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-68d6459fbb-r2mqj"
Jul 06 18:27:27.214976 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:27.214951    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-68d6459fbb-r2mqj"]
Jul 06 18:27:27.217029 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:27:27.216966    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podddfe9a5b_4ef3_4bad_8c95_9ab633bf51a2.slice/crio-e65523c27792a15408e3ce0d4bcf7d2a5370996f23d05423e100e0ca40710486 WatchSource:0}: Error finding container e65523c27792a15408e3ce0d4bcf7d2a5370996f23d05423e100e0ca40710486: Status 404 returned error can't find the container with id e65523c27792a15408e3ce0d4bcf7d2a5370996f23d05423e100e0ca40710486
Jul 06 18:27:27.686188 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:27.686143    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-68d6459fbb-r2mqj" event={"ID":"ddfe9a5b-4ef3-4bad-8c95-9ab633bf51a2","Type":"ContainerStarted","Data":"e65523c27792a15408e3ce0d4bcf7d2a5370996f23d05423e100e0ca40710486"}
Jul 06 18:27:27.687629 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:27.687601    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6d7fb587f8-kdcpk" event={"ID":"9a79f49f-e2d2-4873-9316-91e71019f330","Type":"ContainerStarted","Data":"80d6d232853cfae565beee128667795ce8d9377f6ed4ff026222ac9cd8e6831e"}
Jul 06 18:27:27.687765 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:27.687638    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6d7fb587f8-kdcpk" event={"ID":"9a79f49f-e2d2-4873-9316-91e71019f330","Type":"ContainerStarted","Data":"3b6c0e35ff065f07b5848376c387c811b31bf11fe9e9039c9188edd1cfda26b3"}
Jul 06 18:27:27.687765 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:27.687660    2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-6d7fb587f8-kdcpk" podUID="9a79f49f-e2d2-4873-9316-91e71019f330" containerName="authorino" containerID="cri-o://80d6d232853cfae565beee128667795ce8d9377f6ed4ff026222ac9cd8e6831e" gracePeriod=30
Jul 06 18:27:27.688991 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:27.688966    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-8stt9" event={"ID":"09d090f5-aea9-4603-b50e-91e490684569","Type":"ContainerStarted","Data":"8e7ebc03fe085018d1fd94b943309e4f2407d3c3ab8d328c140edfe26ff71d0c"}
Jul 06 18:27:27.689136 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:27.689106    2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-5d4b67c446-8stt9" podUID="09d090f5-aea9-4603-b50e-91e490684569" containerName="authorino" containerID="cri-o://8e7ebc03fe085018d1fd94b943309e4f2407d3c3ab8d328c140edfe26ff71d0c" gracePeriod=30
Jul 06 18:27:27.704595 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:27.704551    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-6d7fb587f8-kdcpk" podStartSLOduration=1.22091769 podStartE2EDuration="1.704536771s" totalImagesPullingTime="483.619081ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:27:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:27:27.703065813 +0000 UTC m=+582.860464299" watchObservedRunningTime="2026-07-06 18:27:27.704536771 +0000 UTC m=+582.861935298"
Jul 06 18:27:27.721401 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:27.721351    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d4b67c446-8stt9" podStartSLOduration=1.395622494 podStartE2EDuration="1.721334882s" totalImagesPullingTime="325.712388ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:27:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:27:27.720227677 +0000 UTC m=+582.877626166" watchObservedRunningTime="2026-07-06 18:27:27.721334882 +0000 UTC m=+582.878733405"
Jul 06 18:27:27.951257 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:27.951233    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d7fb587f8-kdcpk"
Jul 06 18:27:27.951750 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:27.951731    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z9bvc\" (UniqueName: \"kubernetes.io/projected/9a79f49f-e2d2-4873-9316-91e71019f330-kube-api-access-z9bvc\") pod \"9a79f49f-e2d2-4873-9316-91e71019f330\" (UID: \"9a79f49f-e2d2-4873-9316-91e71019f330\") "
Jul 06 18:27:27.953617 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:27.953589    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9a79f49f-e2d2-4873-9316-91e71019f330-kube-api-access-z9bvc" (OuterVolumeSpecName: "kube-api-access-z9bvc") pod "9a79f49f-e2d2-4873-9316-91e71019f330" (UID: "9a79f49f-e2d2-4873-9316-91e71019f330"). InnerVolumeSpecName "kube-api-access-z9bvc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 18:27:27.965666 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:27.965645    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-8stt9"
Jul 06 18:27:28.052881 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:28.052849    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ps64q\" (UniqueName: \"kubernetes.io/projected/09d090f5-aea9-4603-b50e-91e490684569-kube-api-access-ps64q\") pod \"09d090f5-aea9-4603-b50e-91e490684569\" (UID: \"09d090f5-aea9-4603-b50e-91e490684569\") "
Jul 06 18:27:28.053083 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:28.053069    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z9bvc\" (UniqueName: \"kubernetes.io/projected/9a79f49f-e2d2-4873-9316-91e71019f330-kube-api-access-z9bvc\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:27:28.054908 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:28.054883    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/09d090f5-aea9-4603-b50e-91e490684569-kube-api-access-ps64q" (OuterVolumeSpecName: "kube-api-access-ps64q") pod "09d090f5-aea9-4603-b50e-91e490684569" (UID: "09d090f5-aea9-4603-b50e-91e490684569"). InnerVolumeSpecName "kube-api-access-ps64q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 18:27:28.153878 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:28.153840    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ps64q\" (UniqueName: \"kubernetes.io/projected/09d090f5-aea9-4603-b50e-91e490684569-kube-api-access-ps64q\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:27:28.695308 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:28.695195    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-68d6459fbb-r2mqj" event={"ID":"ddfe9a5b-4ef3-4bad-8c95-9ab633bf51a2","Type":"ContainerStarted","Data":"6d01028ada7be11aef834293db68a23759de5ff0b856987aba0ca5f1e2d31e6b"}
Jul 06 18:27:28.696351 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:28.696319    2580 generic.go:356] "Generic (PLEG): container finished" podID="9a79f49f-e2d2-4873-9316-91e71019f330" containerID="80d6d232853cfae565beee128667795ce8d9377f6ed4ff026222ac9cd8e6831e" exitCode=0
Jul 06 18:27:28.696468 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:28.696354    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d7fb587f8-kdcpk"
Jul 06 18:27:28.696468 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:28.696400    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6d7fb587f8-kdcpk" event={"ID":"9a79f49f-e2d2-4873-9316-91e71019f330","Type":"ContainerDied","Data":"80d6d232853cfae565beee128667795ce8d9377f6ed4ff026222ac9cd8e6831e"}
Jul 06 18:27:28.696468 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:28.696431    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6d7fb587f8-kdcpk" event={"ID":"9a79f49f-e2d2-4873-9316-91e71019f330","Type":"ContainerDied","Data":"3b6c0e35ff065f07b5848376c387c811b31bf11fe9e9039c9188edd1cfda26b3"}
Jul 06 18:27:28.696468 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:28.696450    2580 scope.go:117] "RemoveContainer" containerID="80d6d232853cfae565beee128667795ce8d9377f6ed4ff026222ac9cd8e6831e"
Jul 06 18:27:28.697703 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:28.697671    2580 generic.go:356] "Generic (PLEG): container finished" podID="09d090f5-aea9-4603-b50e-91e490684569" containerID="8e7ebc03fe085018d1fd94b943309e4f2407d3c3ab8d328c140edfe26ff71d0c" exitCode=0
Jul 06 18:27:28.697760 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:28.697712    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-8stt9"
Jul 06 18:27:28.697808 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:28.697752    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-8stt9" event={"ID":"09d090f5-aea9-4603-b50e-91e490684569","Type":"ContainerDied","Data":"8e7ebc03fe085018d1fd94b943309e4f2407d3c3ab8d328c140edfe26ff71d0c"}
Jul 06 18:27:28.697808 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:28.697789    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-8stt9" event={"ID":"09d090f5-aea9-4603-b50e-91e490684569","Type":"ContainerDied","Data":"02fbc2efa5bf23db83fb92d8301e8588989eb47db86c1bce2209ed1edc8723ff"}
Jul 06 18:27:28.706425 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:28.706403    2580 scope.go:117] "RemoveContainer" containerID="80d6d232853cfae565beee128667795ce8d9377f6ed4ff026222ac9cd8e6831e"
Jul 06 18:27:28.706726 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:27:28.706708    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"80d6d232853cfae565beee128667795ce8d9377f6ed4ff026222ac9cd8e6831e\": container with ID starting with 80d6d232853cfae565beee128667795ce8d9377f6ed4ff026222ac9cd8e6831e not found: ID does not exist" containerID="80d6d232853cfae565beee128667795ce8d9377f6ed4ff026222ac9cd8e6831e"
Jul 06 18:27:28.706775 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:28.706737    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"80d6d232853cfae565beee128667795ce8d9377f6ed4ff026222ac9cd8e6831e"} err="failed to get container status \"80d6d232853cfae565beee128667795ce8d9377f6ed4ff026222ac9cd8e6831e\": rpc error: code = NotFound desc = could not find container \"80d6d232853cfae565beee128667795ce8d9377f6ed4ff026222ac9cd8e6831e\": container with ID starting with 80d6d232853cfae565beee128667795ce8d9377f6ed4ff026222ac9cd8e6831e not found: ID does not exist"
Jul 06 18:27:28.706775 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:28.706756    2580 scope.go:117] "RemoveContainer" containerID="8e7ebc03fe085018d1fd94b943309e4f2407d3c3ab8d328c140edfe26ff71d0c"
Jul 06 18:27:28.713400 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:28.713361    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-68d6459fbb-r2mqj" podStartSLOduration=2.276669264 podStartE2EDuration="2.713344424s" totalImagesPullingTime="436.67516ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:27:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:27:28.712479648 +0000 UTC m=+583.869878150" watchObservedRunningTime="2026-07-06 18:27:28.713344424 +0000 UTC m=+583.870742910"
Jul 06 18:27:28.714987 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:28.714967    2580 scope.go:117] "RemoveContainer" containerID="8e7ebc03fe085018d1fd94b943309e4f2407d3c3ab8d328c140edfe26ff71d0c"
Jul 06 18:27:28.715278 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:27:28.715261    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8e7ebc03fe085018d1fd94b943309e4f2407d3c3ab8d328c140edfe26ff71d0c\": container with ID starting with 8e7ebc03fe085018d1fd94b943309e4f2407d3c3ab8d328c140edfe26ff71d0c not found: ID does not exist" containerID="8e7ebc03fe085018d1fd94b943309e4f2407d3c3ab8d328c140edfe26ff71d0c"
Jul 06 18:27:28.715344 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:28.715286    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8e7ebc03fe085018d1fd94b943309e4f2407d3c3ab8d328c140edfe26ff71d0c"} err="failed to get container status \"8e7ebc03fe085018d1fd94b943309e4f2407d3c3ab8d328c140edfe26ff71d0c\": rpc error: code = NotFound desc = could not find container \"8e7ebc03fe085018d1fd94b943309e4f2407d3c3ab8d328c140edfe26ff71d0c\": container with ID starting with 8e7ebc03fe085018d1fd94b943309e4f2407d3c3ab8d328c140edfe26ff71d0c not found: ID does not exist"
Jul 06 18:27:28.729104 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:28.729076    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-8stt9"]
Jul 06 18:27:28.734915 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:28.734891    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-8stt9"]
Jul 06 18:27:28.755712 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:28.755679    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6d7fb587f8-kdcpk"]
Jul 06 18:27:28.766624 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:28.766592    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6d7fb587f8-kdcpk"]
Jul 06 18:27:29.324254 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:29.324221    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="09d090f5-aea9-4603-b50e-91e490684569" path="/var/lib/kubelet/pods/09d090f5-aea9-4603-b50e-91e490684569/volumes"
Jul 06 18:27:29.324630 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:29.324528    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9a79f49f-e2d2-4873-9316-91e71019f330" path="/var/lib/kubelet/pods/9a79f49f-e2d2-4873-9316-91e71019f330/volumes"
Jul 06 18:27:33.183252 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:33.183215    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-56c4d9dd76-42hgq"]
Jul 06 18:27:33.183772 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:33.183749    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="09d090f5-aea9-4603-b50e-91e490684569" containerName="authorino"
Jul 06 18:27:33.183842 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:33.183775    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="09d090f5-aea9-4603-b50e-91e490684569" containerName="authorino"
Jul 06 18:27:33.183842 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:33.183794    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9a79f49f-e2d2-4873-9316-91e71019f330" containerName="authorino"
Jul 06 18:27:33.183842 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:33.183803    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="9a79f49f-e2d2-4873-9316-91e71019f330" containerName="authorino"
Jul 06 18:27:33.184000 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:33.183883    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="9a79f49f-e2d2-4873-9316-91e71019f330" containerName="authorino"
Jul 06 18:27:33.184000 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:33.183899    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="09d090f5-aea9-4603-b50e-91e490684569" containerName="authorino"
Jul 06 18:27:33.270122 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:33.270090    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-56c4d9dd76-42hgq"]
Jul 06 18:27:33.270292 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:33.270210    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-56c4d9dd76-42hgq"
Jul 06 18:27:33.274344 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:33.274325    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 06 18:27:33.274493 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:33.274329    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-cxmll\""
Jul 06 18:27:33.274493 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:33.274332    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 06 18:27:33.298301 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:33.298275    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8pzws\" (UniqueName: \"kubernetes.io/projected/0b251f60-cc96-4b16-b63e-239e888ef369-kube-api-access-8pzws\") pod \"maas-controller-56c4d9dd76-42hgq\" (UID: \"0b251f60-cc96-4b16-b63e-239e888ef369\") " pod="opendatahub/maas-controller-56c4d9dd76-42hgq"
Jul 06 18:27:33.298426 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:33.298317    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0b251f60-cc96-4b16-b63e-239e888ef369-webhook-cert\") pod \"maas-controller-56c4d9dd76-42hgq\" (UID: \"0b251f60-cc96-4b16-b63e-239e888ef369\") " pod="opendatahub/maas-controller-56c4d9dd76-42hgq"
Jul 06 18:27:33.358711 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:33.358675    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-c84d79b86-v95hp"]
Jul 06 18:27:33.399544 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:33.399505    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8pzws\" (UniqueName: \"kubernetes.io/projected/0b251f60-cc96-4b16-b63e-239e888ef369-kube-api-access-8pzws\") pod \"maas-controller-56c4d9dd76-42hgq\" (UID: \"0b251f60-cc96-4b16-b63e-239e888ef369\") " pod="opendatahub/maas-controller-56c4d9dd76-42hgq"
Jul 06 18:27:33.399688 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:33.399553    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0b251f60-cc96-4b16-b63e-239e888ef369-webhook-cert\") pod \"maas-controller-56c4d9dd76-42hgq\" (UID: \"0b251f60-cc96-4b16-b63e-239e888ef369\") " pod="opendatahub/maas-controller-56c4d9dd76-42hgq"
Jul 06 18:27:33.400798 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:33.400775    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-c84d79b86-v95hp"]
Jul 06 18:27:33.400923 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:33.400857    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-c84d79b86-v95hp"
Jul 06 18:27:33.401998 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:33.401973    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0b251f60-cc96-4b16-b63e-239e888ef369-webhook-cert\") pod \"maas-controller-56c4d9dd76-42hgq\" (UID: \"0b251f60-cc96-4b16-b63e-239e888ef369\") " pod="opendatahub/maas-controller-56c4d9dd76-42hgq"
Jul 06 18:27:33.407808 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:33.407787    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8pzws\" (UniqueName: \"kubernetes.io/projected/0b251f60-cc96-4b16-b63e-239e888ef369-kube-api-access-8pzws\") pod \"maas-controller-56c4d9dd76-42hgq\" (UID: \"0b251f60-cc96-4b16-b63e-239e888ef369\") " pod="opendatahub/maas-controller-56c4d9dd76-42hgq"
Jul 06 18:27:33.500059 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:33.499956    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6qptf\" (UniqueName: \"kubernetes.io/projected/9ab48cb5-c3b3-428e-b6b8-d7fae95ef3b2-kube-api-access-6qptf\") pod \"maas-controller-c84d79b86-v95hp\" (UID: \"9ab48cb5-c3b3-428e-b6b8-d7fae95ef3b2\") " pod="opendatahub/maas-controller-c84d79b86-v95hp"
Jul 06 18:27:33.500059 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:33.500045    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9ab48cb5-c3b3-428e-b6b8-d7fae95ef3b2-webhook-cert\") pod \"maas-controller-c84d79b86-v95hp\" (UID: \"9ab48cb5-c3b3-428e-b6b8-d7fae95ef3b2\") " pod="opendatahub/maas-controller-c84d79b86-v95hp"
Jul 06 18:27:33.580100 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:33.580069    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-56c4d9dd76-42hgq"
Jul 06 18:27:33.600659 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:33.600629    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9ab48cb5-c3b3-428e-b6b8-d7fae95ef3b2-webhook-cert\") pod \"maas-controller-c84d79b86-v95hp\" (UID: \"9ab48cb5-c3b3-428e-b6b8-d7fae95ef3b2\") " pod="opendatahub/maas-controller-c84d79b86-v95hp"
Jul 06 18:27:33.600792 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:33.600677    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6qptf\" (UniqueName: \"kubernetes.io/projected/9ab48cb5-c3b3-428e-b6b8-d7fae95ef3b2-kube-api-access-6qptf\") pod \"maas-controller-c84d79b86-v95hp\" (UID: \"9ab48cb5-c3b3-428e-b6b8-d7fae95ef3b2\") " pod="opendatahub/maas-controller-c84d79b86-v95hp"
Jul 06 18:27:33.603168 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:33.603141    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9ab48cb5-c3b3-428e-b6b8-d7fae95ef3b2-webhook-cert\") pod \"maas-controller-c84d79b86-v95hp\" (UID: \"9ab48cb5-c3b3-428e-b6b8-d7fae95ef3b2\") " pod="opendatahub/maas-controller-c84d79b86-v95hp"
Jul 06 18:27:33.609534 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:33.609512    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6qptf\" (UniqueName: \"kubernetes.io/projected/9ab48cb5-c3b3-428e-b6b8-d7fae95ef3b2-kube-api-access-6qptf\") pod \"maas-controller-c84d79b86-v95hp\" (UID: \"9ab48cb5-c3b3-428e-b6b8-d7fae95ef3b2\") " pod="opendatahub/maas-controller-c84d79b86-v95hp"
Jul 06 18:27:33.724851 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:33.724825    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-c84d79b86-v95hp"
Jul 06 18:27:33.730800 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:33.730777    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-56c4d9dd76-42hgq"]
Jul 06 18:27:33.732911 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:27:33.732886    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b251f60_cc96_4b16_b63e_239e888ef369.slice/crio-03d7de39a9c57dd53ba3d486170f99c0f24911bfd62ab309d17d71923ac4a1ab WatchSource:0}: Error finding container 03d7de39a9c57dd53ba3d486170f99c0f24911bfd62ab309d17d71923ac4a1ab: Status 404 returned error can't find the container with id 03d7de39a9c57dd53ba3d486170f99c0f24911bfd62ab309d17d71923ac4a1ab
Jul 06 18:27:33.880133 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:33.880110    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-c84d79b86-v95hp"]
Jul 06 18:27:33.881880 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:27:33.881845    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ab48cb5_c3b3_428e_b6b8_d7fae95ef3b2.slice/crio-2107f07fcd9b9fbf6d8bbfc6732977411b3c9a112e54aa563919717065c7ea00 WatchSource:0}: Error finding container 2107f07fcd9b9fbf6d8bbfc6732977411b3c9a112e54aa563919717065c7ea00: Status 404 returned error can't find the container with id 2107f07fcd9b9fbf6d8bbfc6732977411b3c9a112e54aa563919717065c7ea00
Jul 06 18:27:34.729762 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:34.729715    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-56c4d9dd76-42hgq" event={"ID":"0b251f60-cc96-4b16-b63e-239e888ef369","Type":"ContainerStarted","Data":"03d7de39a9c57dd53ba3d486170f99c0f24911bfd62ab309d17d71923ac4a1ab"}
Jul 06 18:27:34.731217 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:34.731190    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-c84d79b86-v95hp" event={"ID":"9ab48cb5-c3b3-428e-b6b8-d7fae95ef3b2","Type":"ContainerStarted","Data":"2107f07fcd9b9fbf6d8bbfc6732977411b3c9a112e54aa563919717065c7ea00"}
Jul 06 18:27:36.744791 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:36.744730    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-56c4d9dd76-42hgq" event={"ID":"0b251f60-cc96-4b16-b63e-239e888ef369","Type":"ContainerStarted","Data":"1b3c08cd64f6051c95aee9c5439d5db32e74b674827f5ccf633abc2ec58a0244"}
Jul 06 18:27:36.745215 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:36.744844    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-56c4d9dd76-42hgq"
Jul 06 18:27:36.746187 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:36.746158    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-c84d79b86-v95hp" event={"ID":"9ab48cb5-c3b3-428e-b6b8-d7fae95ef3b2","Type":"ContainerStarted","Data":"2e2cc4c4daf44dd55b2740b3ea8aad6e1392ef147914ce9a003c424ff30b01ff"}
Jul 06 18:27:36.746307 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:36.746289    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-c84d79b86-v95hp"
Jul 06 18:27:36.785755 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:36.785701    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-56c4d9dd76-42hgq" podStartSLOduration=0.976777876 podStartE2EDuration="3.785687028s" totalImagesPullingTime="2.808909152s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:27:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:27:36.783764203 +0000 UTC m=+591.941162683" watchObservedRunningTime="2026-07-06 18:27:36.785687028 +0000 UTC m=+591.943085524"
Jul 06 18:27:36.822951 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:36.822902    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-c84d79b86-v95hp" podStartSLOduration=1.158614259 podStartE2EDuration="3.822880331s" totalImagesPullingTime="2.664266072s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:27:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:27:36.82029473 +0000 UTC m=+591.977693215" watchObservedRunningTime="2026-07-06 18:27:36.822880331 +0000 UTC m=+591.980278821"
Jul 06 18:27:47.757691 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:47.757654    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-c84d79b86-v95hp"
Jul 06 18:27:47.758793 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:47.758772    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-56c4d9dd76-42hgq"
Jul 06 18:27:47.845616 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:47.845583    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-56c4d9dd76-42hgq"]
Jul 06 18:27:47.845845 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:47.845818    2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-56c4d9dd76-42hgq" podUID="0b251f60-cc96-4b16-b63e-239e888ef369" containerName="manager" containerID="cri-o://1b3c08cd64f6051c95aee9c5439d5db32e74b674827f5ccf633abc2ec58a0244" gracePeriod=30
Jul 06 18:27:48.097633 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:48.097610    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-56c4d9dd76-42hgq"
Jul 06 18:27:48.224333 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:48.224291    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0b251f60-cc96-4b16-b63e-239e888ef369-webhook-cert\") pod \"0b251f60-cc96-4b16-b63e-239e888ef369\" (UID: \"0b251f60-cc96-4b16-b63e-239e888ef369\") "
Jul 06 18:27:48.224485 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:48.224383    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8pzws\" (UniqueName: \"kubernetes.io/projected/0b251f60-cc96-4b16-b63e-239e888ef369-kube-api-access-8pzws\") pod \"0b251f60-cc96-4b16-b63e-239e888ef369\" (UID: \"0b251f60-cc96-4b16-b63e-239e888ef369\") "
Jul 06 18:27:48.226636 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:48.226607    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0b251f60-cc96-4b16-b63e-239e888ef369-kube-api-access-8pzws" (OuterVolumeSpecName: "kube-api-access-8pzws") pod "0b251f60-cc96-4b16-b63e-239e888ef369" (UID: "0b251f60-cc96-4b16-b63e-239e888ef369"). InnerVolumeSpecName "kube-api-access-8pzws". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 18:27:48.226636 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:48.226611    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0b251f60-cc96-4b16-b63e-239e888ef369-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "0b251f60-cc96-4b16-b63e-239e888ef369" (UID: "0b251f60-cc96-4b16-b63e-239e888ef369"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 18:27:48.325107 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:48.325079    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8pzws\" (UniqueName: \"kubernetes.io/projected/0b251f60-cc96-4b16-b63e-239e888ef369-kube-api-access-8pzws\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:27:48.325107 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:48.325106    2580 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0b251f60-cc96-4b16-b63e-239e888ef369-webhook-cert\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:27:48.798615 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:48.798513    2580 generic.go:356] "Generic (PLEG): container finished" podID="0b251f60-cc96-4b16-b63e-239e888ef369" containerID="1b3c08cd64f6051c95aee9c5439d5db32e74b674827f5ccf633abc2ec58a0244" exitCode=0
Jul 06 18:27:48.798615 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:48.798556    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-56c4d9dd76-42hgq"
Jul 06 18:27:48.798615 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:48.798598    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-56c4d9dd76-42hgq" event={"ID":"0b251f60-cc96-4b16-b63e-239e888ef369","Type":"ContainerDied","Data":"1b3c08cd64f6051c95aee9c5439d5db32e74b674827f5ccf633abc2ec58a0244"}
Jul 06 18:27:48.799092 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:48.798656    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-56c4d9dd76-42hgq" event={"ID":"0b251f60-cc96-4b16-b63e-239e888ef369","Type":"ContainerDied","Data":"03d7de39a9c57dd53ba3d486170f99c0f24911bfd62ab309d17d71923ac4a1ab"}
Jul 06 18:27:48.799092 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:48.798679    2580 scope.go:117] "RemoveContainer" containerID="1b3c08cd64f6051c95aee9c5439d5db32e74b674827f5ccf633abc2ec58a0244"
Jul 06 18:27:48.807215 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:48.807201    2580 scope.go:117] "RemoveContainer" containerID="1b3c08cd64f6051c95aee9c5439d5db32e74b674827f5ccf633abc2ec58a0244"
Jul 06 18:27:48.807459 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:27:48.807440    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1b3c08cd64f6051c95aee9c5439d5db32e74b674827f5ccf633abc2ec58a0244\": container with ID starting with 1b3c08cd64f6051c95aee9c5439d5db32e74b674827f5ccf633abc2ec58a0244 not found: ID does not exist" containerID="1b3c08cd64f6051c95aee9c5439d5db32e74b674827f5ccf633abc2ec58a0244"
Jul 06 18:27:48.807509 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:48.807468    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1b3c08cd64f6051c95aee9c5439d5db32e74b674827f5ccf633abc2ec58a0244"} err="failed to get container status \"1b3c08cd64f6051c95aee9c5439d5db32e74b674827f5ccf633abc2ec58a0244\": rpc error: code = NotFound desc = could not find container \"1b3c08cd64f6051c95aee9c5439d5db32e74b674827f5ccf633abc2ec58a0244\": container with ID starting with 1b3c08cd64f6051c95aee9c5439d5db32e74b674827f5ccf633abc2ec58a0244 not found: ID does not exist"
Jul 06 18:27:48.824842 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:48.824818    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-56c4d9dd76-42hgq"]
Jul 06 18:27:48.829296 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:48.829273    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-56c4d9dd76-42hgq"]
Jul 06 18:27:49.322710 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:27:49.322676    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0b251f60-cc96-4b16-b63e-239e888ef369" path="/var/lib/kubelet/pods/0b251f60-cc96-4b16-b63e-239e888ef369/volumes"
Jul 06 18:28:08.685616 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:28:08.685533    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-79c74f6bbb-cbdff"]
Jul 06 18:28:08.686107 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:28:08.685884    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0b251f60-cc96-4b16-b63e-239e888ef369" containerName="manager"
Jul 06 18:28:08.686107 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:28:08.685895    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="0b251f60-cc96-4b16-b63e-239e888ef369" containerName="manager"
Jul 06 18:28:08.686107 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:28:08.685972    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="0b251f60-cc96-4b16-b63e-239e888ef369" containerName="manager"
Jul 06 18:28:08.690758 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:28:08.690738    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-79c74f6bbb-cbdff"
Jul 06 18:28:08.693742 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:28:08.693710    2580 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"maas-api-dockercfg-v4kjf\" is forbidden: User \"system:node:ip-10-0-134-113.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-134-113.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-v4kjf\"" type="*v1.Secret"
Jul 06 18:28:08.693997 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:28:08.693974    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"maas-api-79c74f6bbb-cbdff\" is forbidden: User \"system:node:ip-10-0-134-113.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-134-113.ec2.internal' and this object" podUID="8ba23aff-6404-4870-8919-6dc1ca89fa0c" pod="opendatahub/maas-api-79c74f6bbb-cbdff"
Jul 06 18:28:08.694275 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:28:08.694257    2580 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"maas-api-serving-cert\" is forbidden: User \"system:node:ip-10-0-134-113.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-134-113.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"opendatahub\"/\"maas-api-serving-cert\"" type="*v1.Secret"
Jul 06 18:28:08.716216 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:28:08.716178    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-79c74f6bbb-cbdff"]
Jul 06 18:28:08.796676 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:28:08.796639    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8ba23aff-6404-4870-8919-6dc1ca89fa0c-maas-api-tls\") pod \"maas-api-79c74f6bbb-cbdff\" (UID: \"8ba23aff-6404-4870-8919-6dc1ca89fa0c\") " pod="opendatahub/maas-api-79c74f6bbb-cbdff"
Jul 06 18:28:08.796676 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:28:08.796679    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8ba23aff-6404-4870-8919-6dc1ca89fa0c-openshift-service-ca\") pod \"maas-api-79c74f6bbb-cbdff\" (UID: \"8ba23aff-6404-4870-8919-6dc1ca89fa0c\") " pod="opendatahub/maas-api-79c74f6bbb-cbdff"
Jul 06 18:28:08.796882 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:28:08.796716    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mlmd8\" (UniqueName: \"kubernetes.io/projected/8ba23aff-6404-4870-8919-6dc1ca89fa0c-kube-api-access-mlmd8\") pod \"maas-api-79c74f6bbb-cbdff\" (UID: \"8ba23aff-6404-4870-8919-6dc1ca89fa0c\") " pod="opendatahub/maas-api-79c74f6bbb-cbdff"
Jul 06 18:28:08.898139 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:28:08.898103    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8ba23aff-6404-4870-8919-6dc1ca89fa0c-maas-api-tls\") pod \"maas-api-79c74f6bbb-cbdff\" (UID: \"8ba23aff-6404-4870-8919-6dc1ca89fa0c\") " pod="opendatahub/maas-api-79c74f6bbb-cbdff"
Jul 06 18:28:08.898317 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:28:08.898148    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8ba23aff-6404-4870-8919-6dc1ca89fa0c-openshift-service-ca\") pod \"maas-api-79c74f6bbb-cbdff\" (UID: \"8ba23aff-6404-4870-8919-6dc1ca89fa0c\") " pod="opendatahub/maas-api-79c74f6bbb-cbdff"
Jul 06 18:28:08.898317 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:28:08.898193    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mlmd8\" (UniqueName: \"kubernetes.io/projected/8ba23aff-6404-4870-8919-6dc1ca89fa0c-kube-api-access-mlmd8\") pod \"maas-api-79c74f6bbb-cbdff\" (UID: \"8ba23aff-6404-4870-8919-6dc1ca89fa0c\") " pod="opendatahub/maas-api-79c74f6bbb-cbdff"
Jul 06 18:28:08.898917 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:28:08.898892    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8ba23aff-6404-4870-8919-6dc1ca89fa0c-openshift-service-ca\") pod \"maas-api-79c74f6bbb-cbdff\" (UID: \"8ba23aff-6404-4870-8919-6dc1ca89fa0c\") " pod="opendatahub/maas-api-79c74f6bbb-cbdff"
Jul 06 18:28:08.906437 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:28:08.906409    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mlmd8\" (UniqueName: \"kubernetes.io/projected/8ba23aff-6404-4870-8919-6dc1ca89fa0c-kube-api-access-mlmd8\") pod \"maas-api-79c74f6bbb-cbdff\" (UID: \"8ba23aff-6404-4870-8919-6dc1ca89fa0c\") " pod="opendatahub/maas-api-79c74f6bbb-cbdff"
Jul 06 18:28:09.192325 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:28:09.192288    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-59ldh"]
Jul 06 18:28:09.195671 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:28:09.195651    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-59ldh"
Jul 06 18:28:09.198301 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:28:09.198281    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-tt62t\""
Jul 06 18:28:09.199006 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:28:09.198988    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Jul 06 18:28:09.210203 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:28:09.210180    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-59ldh"]
Jul 06 18:28:09.301595 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:28:09.301558    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e46693cd-0f3e-4ed5-a5f5-748daad00f43-plugins-config-volume\") pod \"payload-pre-processing-7964457696-59ldh\" (UID: \"e46693cd-0f3e-4ed5-a5f5-748daad00f43\") " pod="openshift-ingress/payload-pre-processing-7964457696-59ldh"
Jul 06 18:28:09.301803 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:28:09.301708    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zcpzm\" (UniqueName: \"kubernetes.io/projected/e46693cd-0f3e-4ed5-a5f5-748daad00f43-kube-api-access-zcpzm\") pod \"payload-pre-processing-7964457696-59ldh\" (UID: \"e46693cd-0f3e-4ed5-a5f5-748daad00f43\") " pod="openshift-ingress/payload-pre-processing-7964457696-59ldh"
Jul 06 18:28:09.402552 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:28:09.402515    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zcpzm\" (UniqueName: \"kubernetes.io/projected/e46693cd-0f3e-4ed5-a5f5-748daad00f43-kube-api-access-zcpzm\") pod \"payload-pre-processing-7964457696-59ldh\" (UID: \"e46693cd-0f3e-4ed5-a5f5-748daad00f43\") " pod="openshift-ingress/payload-pre-processing-7964457696-59ldh"
Jul 06 18:28:09.402713 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:28:09.402577    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e46693cd-0f3e-4ed5-a5f5-748daad00f43-plugins-config-volume\") pod \"payload-pre-processing-7964457696-59ldh\" (UID: \"e46693cd-0f3e-4ed5-a5f5-748daad00f43\") " pod="openshift-ingress/payload-pre-processing-7964457696-59ldh"
Jul 06 18:28:09.403317 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:28:09.403298    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e46693cd-0f3e-4ed5-a5f5-748daad00f43-plugins-config-volume\") pod \"payload-pre-processing-7964457696-59ldh\" (UID: \"e46693cd-0f3e-4ed5-a5f5-748daad00f43\") " pod="openshift-ingress/payload-pre-processing-7964457696-59ldh"
Jul 06 18:28:09.411922 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:28:09.411892    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zcpzm\" (UniqueName: \"kubernetes.io/projected/e46693cd-0f3e-4ed5-a5f5-748daad00f43-kube-api-access-zcpzm\") pod \"payload-pre-processing-7964457696-59ldh\" (UID: \"e46693cd-0f3e-4ed5-a5f5-748daad00f43\") " pod="openshift-ingress/payload-pre-processing-7964457696-59ldh"
Jul 06 18:28:09.506748 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:28:09.506658    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-59ldh"
Jul 06 18:28:09.584610 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:28:09.584579    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-serving-cert\""
Jul 06 18:28:09.592368 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:28:09.592341    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8ba23aff-6404-4870-8919-6dc1ca89fa0c-maas-api-tls\") pod \"maas-api-79c74f6bbb-cbdff\" (UID: \"8ba23aff-6404-4870-8919-6dc1ca89fa0c\") " pod="opendatahub/maas-api-79c74f6bbb-cbdff"
Jul 06 18:28:09.639289 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:28:09.639262    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-59ldh"]
Jul 06 18:28:09.640982 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:28:09.640952    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode46693cd_0f3e_4ed5_a5f5_748daad00f43.slice/crio-96277c1f81ff37471e227f4a4ef5b7674af2b902fac491b9af79f2f3b5530038 WatchSource:0}: Error finding container 96277c1f81ff37471e227f4a4ef5b7674af2b902fac491b9af79f2f3b5530038: Status 404 returned error can't find the container with id 96277c1f81ff37471e227f4a4ef5b7674af2b902fac491b9af79f2f3b5530038
Jul 06 18:28:09.798331 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:28:09.798252    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-v4kjf\""
Jul 06 18:28:09.800861 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:28:09.800842    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-79c74f6bbb-cbdff"
Jul 06 18:28:09.892484 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:28:09.892447    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-59ldh" event={"ID":"e46693cd-0f3e-4ed5-a5f5-748daad00f43","Type":"ContainerStarted","Data":"96277c1f81ff37471e227f4a4ef5b7674af2b902fac491b9af79f2f3b5530038"}
Jul 06 18:28:09.975774 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:28:09.975749    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-79c74f6bbb-cbdff"]
Jul 06 18:28:09.978075 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:28:09.978000    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ba23aff_6404_4870_8919_6dc1ca89fa0c.slice/crio-6854e21b8433f20b804617f1ee941f192d460557860e3e7e0e6ebd302eba9cee WatchSource:0}: Error finding container 6854e21b8433f20b804617f1ee941f192d460557860e3e7e0e6ebd302eba9cee: Status 404 returned error can't find the container with id 6854e21b8433f20b804617f1ee941f192d460557860e3e7e0e6ebd302eba9cee
Jul 06 18:28:10.898868 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:28:10.898835    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-79c74f6bbb-cbdff" event={"ID":"8ba23aff-6404-4870-8919-6dc1ca89fa0c","Type":"ContainerStarted","Data":"6854e21b8433f20b804617f1ee941f192d460557860e3e7e0e6ebd302eba9cee"}
Jul 06 18:28:12.911919 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:28:12.911883    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-59ldh" event={"ID":"e46693cd-0f3e-4ed5-a5f5-748daad00f43","Type":"ContainerStarted","Data":"877c085238501d8ca9deb9fd548cd9b3c262e77435322a5e35ea4c273d7b4514"}
Jul 06 18:28:12.912371 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:28:12.911998    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7964457696-59ldh"
Jul 06 18:28:12.913671 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:28:12.913649    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-79c74f6bbb-cbdff" event={"ID":"8ba23aff-6404-4870-8919-6dc1ca89fa0c","Type":"ContainerStarted","Data":"da8364ed8d54283ea4d8a2173364a3e0188f23a0e87f8f23cfc0bccdb2fbdc28"}
Jul 06 18:28:12.913771 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:28:12.913689    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-79c74f6bbb-cbdff"
Jul 06 18:28:12.932957 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:28:12.932896    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7964457696-59ldh" podStartSLOduration=1.680690272 podStartE2EDuration="3.932876661s" totalImagesPullingTime="2.252186389s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:28:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:28:12.930141523 +0000 UTC m=+628.087540010" watchObservedRunningTime="2026-07-06 18:28:12.932876661 +0000 UTC m=+628.090275140"
Jul 06 18:28:12.949149 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:28:12.949085    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-79c74f6bbb-cbdff" podStartSLOduration=2.968623967 podStartE2EDuration="4.949062959s" totalImagesPullingTime="1.980438992s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:28:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:28:12.946537584 +0000 UTC m=+628.103936072" watchObservedRunningTime="2026-07-06 18:28:12.949062959 +0000 UTC m=+628.106461446"
Jul 06 18:28:16.932194 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:28:16.932114    2580 generic.go:356] "Generic (PLEG): container finished" podID="75b7f81b-fdb8-4d09-8317-8c9323960ed5" containerID="6e422c408719f68333aba3d4026d60f0789c2bc4f794fd76de1ecce6c0b6f4e4" exitCode=0
Jul 06 18:28:16.932660 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:28:16.932195    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tlmbg" event={"ID":"75b7f81b-fdb8-4d09-8317-8c9323960ed5","Type":"ContainerDied","Data":"6e422c408719f68333aba3d4026d60f0789c2bc4f794fd76de1ecce6c0b6f4e4"}
Jul 06 18:28:16.932660 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:28:16.932550    2580 scope.go:117] "RemoveContainer" containerID="6e422c408719f68333aba3d4026d60f0789c2bc4f794fd76de1ecce6c0b6f4e4"
Jul 06 18:28:16.933548 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:28:16.933516    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 06 18:28:16.933671 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:28:16.933584    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 06 18:28:16.933671 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:28:16.933629    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 06 18:28:17.943237 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:28:17.942182    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tlmbg" event={"ID":"75b7f81b-fdb8-4d09-8317-8c9323960ed5","Type":"ContainerStarted","Data":"b69f7dd0d9a58448001f05e052a7bfd8f93a0e1d68d37ca9698c32a23bb9c3da"}
Jul 06 18:28:18.764096 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:28:18.764059    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tlmbg"
Jul 06 18:28:18.764320 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:28:18.764297    2580 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-tlmbg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused" start-of-body=
Jul 06 18:28:18.764400 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:28:18.764355    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tlmbg" podUID="75b7f81b-fdb8-4d09-8317-8c9323960ed5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused"
Jul 06 18:28:18.922998 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:28:18.922970    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-79c74f6bbb-cbdff"
Jul 06 18:28:19.764560 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:28:19.764522    2580 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-tlmbg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused" start-of-body=
Jul 06 18:28:19.764952 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:28:19.764592    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tlmbg" podUID="75b7f81b-fdb8-4d09-8317-8c9323960ed5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused"
Jul 06 18:28:20.763977 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:28:20.763942    2580 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-tlmbg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused" start-of-body=
Jul 06 18:28:20.764164 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:28:20.763991    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tlmbg" podUID="75b7f81b-fdb8-4d09-8317-8c9323960ed5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused"
Jul 06 18:28:21.767865 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:28:21.767834    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tlmbg"
Jul 06 18:28:21.768252 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:28:21.768138    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tlmbg"
Jul 06 18:28:21.768895 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:28:21.768880    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-tlmbg"
Jul 06 18:28:23.919409 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:28:23.919367    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-7964457696-59ldh" podUID="e46693cd-0f3e-4ed5-a5f5-748daad00f43" containerName="payload-pre-processing" probeResult="failure" output="dial tcp 10.133.0.41:9004: connect: connection refused"
Jul 06 18:28:27.594617 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:28:27.594582    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-smh9r"]
Jul 06 18:28:27.596968 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:28:27.596951    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-smh9r"
Jul 06 18:28:27.601116 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:28:27.601079    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 06 18:28:27.601267 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:28:27.601128    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-5psm8\""
Jul 06 18:28:27.601267 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:28:27.601079    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 06 18:28:27.601267 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:28:27.601079    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Jul 06 18:28:27.607257 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:28:27.607232    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-smh9r"]
Jul 06 18:28:27.679045 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:28:27.678978    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s5f4j\" (UniqueName: \"kubernetes.io/projected/50799c4d-2fd5-41a5-aa2b-cc1eb776ae9e-kube-api-access-s5f4j\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-smh9r\" (UID: \"50799c4d-2fd5-41a5-aa2b-cc1eb776ae9e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-smh9r"
Jul 06 18:28:27.679228 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:28:27.679085    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/50799c4d-2fd5-41a5-aa2b-cc1eb776ae9e-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-smh9r\" (UID: \"50799c4d-2fd5-41a5-aa2b-cc1eb776ae9e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-smh9r"
Jul 06 18:28:27.679228 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:28:27.679110    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/50799c4d-2fd5-41a5-aa2b-cc1eb776ae9e-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-smh9r\" (UID: \"50799c4d-2fd5-41a5-aa2b-cc1eb776ae9e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-smh9r"
Jul 06 18:28:27.679228 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:28:27.679191    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/50799c4d-2fd5-41a5-aa2b-cc1eb776ae9e-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-smh9r\" (UID: \"50799c4d-2fd5-41a5-aa2b-cc1eb776ae9e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-smh9r"
Jul 06 18:28:27.679380 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:28:27.679232    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/50799c4d-2fd5-41a5-aa2b-cc1eb776ae9e-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-smh9r\" (UID: \"50799c4d-2fd5-41a5-aa2b-cc1eb776ae9e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-smh9r"
Jul 06 18:28:27.679380 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:28:27.679261    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/50799c4d-2fd5-41a5-aa2b-cc1eb776ae9e-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-smh9r\" (UID: \"50799c4d-2fd5-41a5-aa2b-cc1eb776ae9e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-smh9r"
Jul 06 18:28:27.779849 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:28:27.779818    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s5f4j\" (UniqueName: \"kubernetes.io/projected/50799c4d-2fd5-41a5-aa2b-cc1eb776ae9e-kube-api-access-s5f4j\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-smh9r\" (UID: \"50799c4d-2fd5-41a5-aa2b-cc1eb776ae9e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-smh9r"
Jul 06 18:28:27.780059 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:28:27.779868    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/50799c4d-2fd5-41a5-aa2b-cc1eb776ae9e-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-smh9r\" (UID: \"50799c4d-2fd5-41a5-aa2b-cc1eb776ae9e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-smh9r"
Jul 06 18:28:27.780059 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:28:27.779896    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/50799c4d-2fd5-41a5-aa2b-cc1eb776ae9e-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-smh9r\" (UID: \"50799c4d-2fd5-41a5-aa2b-cc1eb776ae9e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-smh9r"
Jul 06 18:28:27.780059 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:28:27.779934    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/50799c4d-2fd5-41a5-aa2b-cc1eb776ae9e-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-smh9r\" (UID: \"50799c4d-2fd5-41a5-aa2b-cc1eb776ae9e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-smh9r"
Jul 06 18:28:27.780059 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:28:27.779962    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/50799c4d-2fd5-41a5-aa2b-cc1eb776ae9e-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-smh9r\" (UID: \"50799c4d-2fd5-41a5-aa2b-cc1eb776ae9e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-smh9r"
Jul 06 18:28:27.780059 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:28:27.779992    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/50799c4d-2fd5-41a5-aa2b-cc1eb776ae9e-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-smh9r\" (UID: \"50799c4d-2fd5-41a5-aa2b-cc1eb776ae9e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-smh9r"
Jul 06 18:28:27.780410 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:28:27.780388    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/50799c4d-2fd5-41a5-aa2b-cc1eb776ae9e-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-smh9r\" (UID: \"50799c4d-2fd5-41a5-aa2b-cc1eb776ae9e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-smh9r"
Jul 06 18:28:27.780490 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:28:27.780430    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/50799c4d-2fd5-41a5-aa2b-cc1eb776ae9e-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-smh9r\" (UID: \"50799c4d-2fd5-41a5-aa2b-cc1eb776ae9e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-smh9r"
Jul 06 18:28:27.780490 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:28:27.780465    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/50799c4d-2fd5-41a5-aa2b-cc1eb776ae9e-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-smh9r\" (UID: \"50799c4d-2fd5-41a5-aa2b-cc1eb776ae9e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-smh9r"
Jul 06 18:28:27.782144 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:28:27.782107    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/50799c4d-2fd5-41a5-aa2b-cc1eb776ae9e-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-smh9r\" (UID: \"50799c4d-2fd5-41a5-aa2b-cc1eb776ae9e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-smh9r"
Jul 06 18:28:27.782425 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:28:27.782406    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/50799c4d-2fd5-41a5-aa2b-cc1eb776ae9e-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-smh9r\" (UID: \"50799c4d-2fd5-41a5-aa2b-cc1eb776ae9e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-smh9r"
Jul 06 18:28:27.787560 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:28:27.787537    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s5f4j\" (UniqueName: \"kubernetes.io/projected/50799c4d-2fd5-41a5-aa2b-cc1eb776ae9e-kube-api-access-s5f4j\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-smh9r\" (UID: \"50799c4d-2fd5-41a5-aa2b-cc1eb776ae9e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-smh9r"
Jul 06 18:28:27.907556 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:28:27.907461    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-smh9r"
Jul 06 18:28:28.048847 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:28:28.048821    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-smh9r"]
Jul 06 18:28:28.050482 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:28:28.050453    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50799c4d_2fd5_41a5_aa2b_cc1eb776ae9e.slice/crio-e51249c32125228ab686943f777c98e57cf13c4e7b4077146d59dac24d9af139 WatchSource:0}: Error finding container e51249c32125228ab686943f777c98e57cf13c4e7b4077146d59dac24d9af139: Status 404 returned error can't find the container with id e51249c32125228ab686943f777c98e57cf13c4e7b4077146d59dac24d9af139
Jul 06 18:28:29.016608 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:28:29.016572    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-smh9r" event={"ID":"50799c4d-2fd5-41a5-aa2b-cc1eb776ae9e","Type":"ContainerStarted","Data":"e51249c32125228ab686943f777c98e57cf13c4e7b4077146d59dac24d9af139"}
Jul 06 18:28:30.022069 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:28:30.022030    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-smh9r" event={"ID":"50799c4d-2fd5-41a5-aa2b-cc1eb776ae9e","Type":"ContainerStarted","Data":"0f94093df0c4ce832d9e1a604f4f047f5c40a7c794c457286ed815c3319cec3a"}
Jul 06 18:28:30.039254 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:28:30.039205    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-smh9r" podStartSLOduration=1.9002927490000001 podStartE2EDuration="3.039189415s" totalImagesPullingTime="1.138896666s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:28:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:28:30.038142707 +0000 UTC m=+645.195541193" watchObservedRunningTime="2026-07-06 18:28:30.039189415 +0000 UTC m=+645.196587892"
Jul 06 18:28:33.919096 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:28:33.919062    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7964457696-59ldh"
Jul 06 18:28:37.907869 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:28:37.907817    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-smh9r"
Jul 06 18:28:37.908429 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:28:37.907897    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-smh9r"
Jul 06 18:28:37.920199 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:28:37.920174    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-smh9r"
Jul 06 18:28:38.069834 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:28:38.069805    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-smh9r"
Jul 06 18:30:00.143235 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:30:00.143198    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-29722710-ldj78"]
Jul 06 18:30:00.146515 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:30:00.146498    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29722710-ldj78"
Jul 06 18:30:00.153393 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:30:00.153363    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29722710-ldj78"]
Jul 06 18:30:00.263233 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:30:00.263190    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cn6ts\" (UniqueName: \"kubernetes.io/projected/44be2b8f-8aac-4423-aeea-03c7d2caea25-kube-api-access-cn6ts\") pod \"maas-api-key-cleanup-29722710-ldj78\" (UID: \"44be2b8f-8aac-4423-aeea-03c7d2caea25\") " pod="opendatahub/maas-api-key-cleanup-29722710-ldj78"
Jul 06 18:30:00.364163 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:30:00.364127    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cn6ts\" (UniqueName: \"kubernetes.io/projected/44be2b8f-8aac-4423-aeea-03c7d2caea25-kube-api-access-cn6ts\") pod \"maas-api-key-cleanup-29722710-ldj78\" (UID: \"44be2b8f-8aac-4423-aeea-03c7d2caea25\") " pod="opendatahub/maas-api-key-cleanup-29722710-ldj78"
Jul 06 18:30:00.373118 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:30:00.373085    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cn6ts\" (UniqueName: \"kubernetes.io/projected/44be2b8f-8aac-4423-aeea-03c7d2caea25-kube-api-access-cn6ts\") pod \"maas-api-key-cleanup-29722710-ldj78\" (UID: \"44be2b8f-8aac-4423-aeea-03c7d2caea25\") " pod="opendatahub/maas-api-key-cleanup-29722710-ldj78"
Jul 06 18:30:00.458750 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:30:00.458661    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29722710-ldj78"
Jul 06 18:30:00.592492 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:30:00.592428    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29722710-ldj78"]
Jul 06 18:30:00.594699 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:30:00.594670    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44be2b8f_8aac_4423_aeea_03c7d2caea25.slice/crio-d93720e048cd1f9214aceb2353111a6491a3ee7053a85ad72d955558fc93afe6 WatchSource:0}: Error finding container d93720e048cd1f9214aceb2353111a6491a3ee7053a85ad72d955558fc93afe6: Status 404 returned error can't find the container with id d93720e048cd1f9214aceb2353111a6491a3ee7053a85ad72d955558fc93afe6
Jul 06 18:30:00.596594 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:30:00.596579    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 06 18:30:01.436558 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:30:01.436515    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29722710-ldj78" event={"ID":"44be2b8f-8aac-4423-aeea-03c7d2caea25","Type":"ContainerStarted","Data":"d93720e048cd1f9214aceb2353111a6491a3ee7053a85ad72d955558fc93afe6"}
Jul 06 18:30:03.447508 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:30:03.447412    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29722710-ldj78" event={"ID":"44be2b8f-8aac-4423-aeea-03c7d2caea25","Type":"ContainerStarted","Data":"798f7d2c47d7d7d80f97dd56af90952bac10bbaf2a1cb7e4be86819e36a76336"}
Jul 06 18:30:03.464057 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:30:03.463986    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-key-cleanup-29722710-ldj78" podStartSLOduration=1.178295025 podStartE2EDuration="3.4639706s" totalImagesPullingTime="2.285675575s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:30:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:30:03.462246071 +0000 UTC m=+738.619644557" watchObservedRunningTime="2026-07-06 18:30:03.4639706 +0000 UTC m=+738.621369086"
Jul 06 18:30:04.453142 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:30:04.453100    2580 generic.go:356] "Generic (PLEG): container finished" podID="44be2b8f-8aac-4423-aeea-03c7d2caea25" containerID="798f7d2c47d7d7d80f97dd56af90952bac10bbaf2a1cb7e4be86819e36a76336" exitCode=0
Jul 06 18:30:04.453142 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:30:04.453143    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29722710-ldj78" event={"ID":"44be2b8f-8aac-4423-aeea-03c7d2caea25","Type":"ContainerDied","Data":"798f7d2c47d7d7d80f97dd56af90952bac10bbaf2a1cb7e4be86819e36a76336"}
Jul 06 18:30:05.587969 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:30:05.587944    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29722710-ldj78"
Jul 06 18:30:05.712400 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:30:05.712309    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cn6ts\" (UniqueName: \"kubernetes.io/projected/44be2b8f-8aac-4423-aeea-03c7d2caea25-kube-api-access-cn6ts\") pod \"44be2b8f-8aac-4423-aeea-03c7d2caea25\" (UID: \"44be2b8f-8aac-4423-aeea-03c7d2caea25\") "
Jul 06 18:30:05.714303 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:30:05.714282    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/44be2b8f-8aac-4423-aeea-03c7d2caea25-kube-api-access-cn6ts" (OuterVolumeSpecName: "kube-api-access-cn6ts") pod "44be2b8f-8aac-4423-aeea-03c7d2caea25" (UID: "44be2b8f-8aac-4423-aeea-03c7d2caea25"). InnerVolumeSpecName "kube-api-access-cn6ts". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 18:30:05.813648 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:30:05.813615    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cn6ts\" (UniqueName: \"kubernetes.io/projected/44be2b8f-8aac-4423-aeea-03c7d2caea25-kube-api-access-cn6ts\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:30:06.462632 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:30:06.462604    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29722710-ldj78"
Jul 06 18:30:06.462799 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:30:06.462600    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29722710-ldj78" event={"ID":"44be2b8f-8aac-4423-aeea-03c7d2caea25","Type":"ContainerDied","Data":"d93720e048cd1f9214aceb2353111a6491a3ee7053a85ad72d955558fc93afe6"}
Jul 06 18:30:06.462799 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:30:06.462733    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d93720e048cd1f9214aceb2353111a6491a3ee7053a85ad72d955558fc93afe6"
Jul 06 18:30:19.778457 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:30:19.778426    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5cb897bc47-8zbh8"]
Jul 06 18:30:19.778820 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:30:19.778797    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="44be2b8f-8aac-4423-aeea-03c7d2caea25" containerName="cleanup"
Jul 06 18:30:19.778820 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:30:19.778808    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="44be2b8f-8aac-4423-aeea-03c7d2caea25" containerName="cleanup"
Jul 06 18:30:19.778891 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:30:19.778867    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="44be2b8f-8aac-4423-aeea-03c7d2caea25" containerName="cleanup"
Jul 06 18:30:19.785146 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:30:19.785121    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5cb897bc47-8zbh8"
Jul 06 18:30:19.793090 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:30:19.792673    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5cb897bc47-8zbh8"]
Jul 06 18:30:19.939577 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:30:19.939538    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wwfpn\" (UniqueName: \"kubernetes.io/projected/b2278607-0062-4bc4-8f3f-57cc9a60f2a4-kube-api-access-wwfpn\") pod \"authorino-5cb897bc47-8zbh8\" (UID: \"b2278607-0062-4bc4-8f3f-57cc9a60f2a4\") " pod="kuadrant-system/authorino-5cb897bc47-8zbh8"
Jul 06 18:30:19.939729 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:30:19.939595    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/b2278607-0062-4bc4-8f3f-57cc9a60f2a4-tls-cert\") pod \"authorino-5cb897bc47-8zbh8\" (UID: \"b2278607-0062-4bc4-8f3f-57cc9a60f2a4\") " pod="kuadrant-system/authorino-5cb897bc47-8zbh8"
Jul 06 18:30:20.040768 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:30:20.040683    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wwfpn\" (UniqueName: \"kubernetes.io/projected/b2278607-0062-4bc4-8f3f-57cc9a60f2a4-kube-api-access-wwfpn\") pod \"authorino-5cb897bc47-8zbh8\" (UID: \"b2278607-0062-4bc4-8f3f-57cc9a60f2a4\") " pod="kuadrant-system/authorino-5cb897bc47-8zbh8"
Jul 06 18:30:20.040768 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:30:20.040738    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/b2278607-0062-4bc4-8f3f-57cc9a60f2a4-tls-cert\") pod \"authorino-5cb897bc47-8zbh8\" (UID: \"b2278607-0062-4bc4-8f3f-57cc9a60f2a4\") " pod="kuadrant-system/authorino-5cb897bc47-8zbh8"
Jul 06 18:30:20.043255 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:30:20.043229    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/b2278607-0062-4bc4-8f3f-57cc9a60f2a4-tls-cert\") pod \"authorino-5cb897bc47-8zbh8\" (UID: \"b2278607-0062-4bc4-8f3f-57cc9a60f2a4\") " pod="kuadrant-system/authorino-5cb897bc47-8zbh8"
Jul 06 18:30:20.048327 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:30:20.048307    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wwfpn\" (UniqueName: \"kubernetes.io/projected/b2278607-0062-4bc4-8f3f-57cc9a60f2a4-kube-api-access-wwfpn\") pod \"authorino-5cb897bc47-8zbh8\" (UID: \"b2278607-0062-4bc4-8f3f-57cc9a60f2a4\") " pod="kuadrant-system/authorino-5cb897bc47-8zbh8"
Jul 06 18:30:20.095282 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:30:20.095246    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5cb897bc47-8zbh8"
Jul 06 18:30:20.236247 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:30:20.236222    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5cb897bc47-8zbh8"]
Jul 06 18:30:20.238698 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:30:20.238661    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb2278607_0062_4bc4_8f3f_57cc9a60f2a4.slice/crio-1dc8c12b151e8ccb50fea1d2a775acfe279eb52e62136c98ee99aa66df5bb717 WatchSource:0}: Error finding container 1dc8c12b151e8ccb50fea1d2a775acfe279eb52e62136c98ee99aa66df5bb717: Status 404 returned error can't find the container with id 1dc8c12b151e8ccb50fea1d2a775acfe279eb52e62136c98ee99aa66df5bb717
Jul 06 18:30:20.523457 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:30:20.523422    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5cb897bc47-8zbh8" event={"ID":"b2278607-0062-4bc4-8f3f-57cc9a60f2a4","Type":"ContainerStarted","Data":"1dc8c12b151e8ccb50fea1d2a775acfe279eb52e62136c98ee99aa66df5bb717"}
Jul 06 18:30:21.532174 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:30:21.532133    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5cb897bc47-8zbh8" event={"ID":"b2278607-0062-4bc4-8f3f-57cc9a60f2a4","Type":"ContainerStarted","Data":"91001c5f5f1671ba7ae239f4fdc5927386d05d2f60f74c8b01b0f9996eaa352a"}
Jul 06 18:30:21.553109 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:30:21.553051    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5cb897bc47-8zbh8" podStartSLOduration=2.138778904 podStartE2EDuration="2.553008673s" totalImagesPullingTime="414.229769ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:30:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:30:21.550974863 +0000 UTC m=+756.708373350" watchObservedRunningTime="2026-07-06 18:30:21.553008673 +0000 UTC m=+756.710407206"
Jul 06 18:30:21.591333 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:30:21.591295    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-68d6459fbb-r2mqj"]
Jul 06 18:30:21.591529 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:30:21.591503    2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-68d6459fbb-r2mqj" podUID="ddfe9a5b-4ef3-4bad-8c95-9ab633bf51a2" containerName="authorino" containerID="cri-o://6d01028ada7be11aef834293db68a23759de5ff0b856987aba0ca5f1e2d31e6b" gracePeriod=30
Jul 06 18:30:21.852559 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:30:21.852521    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-68d6459fbb-r2mqj"
Jul 06 18:30:21.957005 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:30:21.956964    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ddfe9a5b-4ef3-4bad-8c95-9ab633bf51a2-tls-cert\") pod \"ddfe9a5b-4ef3-4bad-8c95-9ab633bf51a2\" (UID: \"ddfe9a5b-4ef3-4bad-8c95-9ab633bf51a2\") "
Jul 06 18:30:21.957178 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:30:21.957082    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xf77p\" (UniqueName: \"kubernetes.io/projected/ddfe9a5b-4ef3-4bad-8c95-9ab633bf51a2-kube-api-access-xf77p\") pod \"ddfe9a5b-4ef3-4bad-8c95-9ab633bf51a2\" (UID: \"ddfe9a5b-4ef3-4bad-8c95-9ab633bf51a2\") "
Jul 06 18:30:21.959136 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:30:21.959101    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ddfe9a5b-4ef3-4bad-8c95-9ab633bf51a2-kube-api-access-xf77p" (OuterVolumeSpecName: "kube-api-access-xf77p") pod "ddfe9a5b-4ef3-4bad-8c95-9ab633bf51a2" (UID: "ddfe9a5b-4ef3-4bad-8c95-9ab633bf51a2"). InnerVolumeSpecName "kube-api-access-xf77p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 18:30:21.968635 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:30:21.968607    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ddfe9a5b-4ef3-4bad-8c95-9ab633bf51a2-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "ddfe9a5b-4ef3-4bad-8c95-9ab633bf51a2" (UID: "ddfe9a5b-4ef3-4bad-8c95-9ab633bf51a2"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 18:30:22.057850 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:30:22.057817    2580 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ddfe9a5b-4ef3-4bad-8c95-9ab633bf51a2-tls-cert\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:30:22.057850 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:30:22.057847    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xf77p\" (UniqueName: \"kubernetes.io/projected/ddfe9a5b-4ef3-4bad-8c95-9ab633bf51a2-kube-api-access-xf77p\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:30:22.539999 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:30:22.539960    2580 generic.go:356] "Generic (PLEG): container finished" podID="ddfe9a5b-4ef3-4bad-8c95-9ab633bf51a2" containerID="6d01028ada7be11aef834293db68a23759de5ff0b856987aba0ca5f1e2d31e6b" exitCode=0
Jul 06 18:30:22.540469 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:30:22.540051    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-68d6459fbb-r2mqj"
Jul 06 18:30:22.540469 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:30:22.540051    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-68d6459fbb-r2mqj" event={"ID":"ddfe9a5b-4ef3-4bad-8c95-9ab633bf51a2","Type":"ContainerDied","Data":"6d01028ada7be11aef834293db68a23759de5ff0b856987aba0ca5f1e2d31e6b"}
Jul 06 18:30:22.540469 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:30:22.540170    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-68d6459fbb-r2mqj" event={"ID":"ddfe9a5b-4ef3-4bad-8c95-9ab633bf51a2","Type":"ContainerDied","Data":"e65523c27792a15408e3ce0d4bcf7d2a5370996f23d05423e100e0ca40710486"}
Jul 06 18:30:22.540469 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:30:22.540190    2580 scope.go:117] "RemoveContainer" containerID="6d01028ada7be11aef834293db68a23759de5ff0b856987aba0ca5f1e2d31e6b"
Jul 06 18:30:22.548561 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:30:22.548542    2580 scope.go:117] "RemoveContainer" containerID="6d01028ada7be11aef834293db68a23759de5ff0b856987aba0ca5f1e2d31e6b"
Jul 06 18:30:22.548843 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:30:22.548819    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6d01028ada7be11aef834293db68a23759de5ff0b856987aba0ca5f1e2d31e6b\": container with ID starting with 6d01028ada7be11aef834293db68a23759de5ff0b856987aba0ca5f1e2d31e6b not found: ID does not exist" containerID="6d01028ada7be11aef834293db68a23759de5ff0b856987aba0ca5f1e2d31e6b"
Jul 06 18:30:22.548913 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:30:22.548851    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6d01028ada7be11aef834293db68a23759de5ff0b856987aba0ca5f1e2d31e6b"} err="failed to get container status \"6d01028ada7be11aef834293db68a23759de5ff0b856987aba0ca5f1e2d31e6b\": rpc error: code = NotFound desc = could not find container \"6d01028ada7be11aef834293db68a23759de5ff0b856987aba0ca5f1e2d31e6b\": container with ID starting with 6d01028ada7be11aef834293db68a23759de5ff0b856987aba0ca5f1e2d31e6b not found: ID does not exist"
Jul 06 18:30:22.565032 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:30:22.564991    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-68d6459fbb-r2mqj"]
Jul 06 18:30:22.575621 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:30:22.575584    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-68d6459fbb-r2mqj"]
Jul 06 18:30:23.323502 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:30:23.323476    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ddfe9a5b-4ef3-4bad-8c95-9ab633bf51a2" path="/var/lib/kubelet/pods/ddfe9a5b-4ef3-4bad-8c95-9ab633bf51a2/volumes"
Jul 06 18:31:34.289389 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:31:34.289309    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-c84d79b86-v95hp"]
Jul 06 18:31:34.290028 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:31:34.289525    2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-c84d79b86-v95hp" podUID="9ab48cb5-c3b3-428e-b6b8-d7fae95ef3b2" containerName="manager" containerID="cri-o://2e2cc4c4daf44dd55b2740b3ea8aad6e1392ef147914ce9a003c424ff30b01ff" gracePeriod=30
Jul 06 18:31:34.541534 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:31:34.541473    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-c84d79b86-v95hp"
Jul 06 18:31:34.579041 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:31:34.578990    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6qptf\" (UniqueName: \"kubernetes.io/projected/9ab48cb5-c3b3-428e-b6b8-d7fae95ef3b2-kube-api-access-6qptf\") pod \"9ab48cb5-c3b3-428e-b6b8-d7fae95ef3b2\" (UID: \"9ab48cb5-c3b3-428e-b6b8-d7fae95ef3b2\") "
Jul 06 18:31:34.579221 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:31:34.579138    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9ab48cb5-c3b3-428e-b6b8-d7fae95ef3b2-webhook-cert\") pod \"9ab48cb5-c3b3-428e-b6b8-d7fae95ef3b2\" (UID: \"9ab48cb5-c3b3-428e-b6b8-d7fae95ef3b2\") "
Jul 06 18:31:34.581178 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:31:34.581145    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9ab48cb5-c3b3-428e-b6b8-d7fae95ef3b2-kube-api-access-6qptf" (OuterVolumeSpecName: "kube-api-access-6qptf") pod "9ab48cb5-c3b3-428e-b6b8-d7fae95ef3b2" (UID: "9ab48cb5-c3b3-428e-b6b8-d7fae95ef3b2"). InnerVolumeSpecName "kube-api-access-6qptf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 18:31:34.581459 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:31:34.581440    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9ab48cb5-c3b3-428e-b6b8-d7fae95ef3b2-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "9ab48cb5-c3b3-428e-b6b8-d7fae95ef3b2" (UID: "9ab48cb5-c3b3-428e-b6b8-d7fae95ef3b2"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 18:31:34.680154 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:31:34.680120    2580 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9ab48cb5-c3b3-428e-b6b8-d7fae95ef3b2-webhook-cert\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:31:34.680154 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:31:34.680148    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6qptf\" (UniqueName: \"kubernetes.io/projected/9ab48cb5-c3b3-428e-b6b8-d7fae95ef3b2-kube-api-access-6qptf\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:31:34.876163 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:31:34.876126    2580 generic.go:356] "Generic (PLEG): container finished" podID="9ab48cb5-c3b3-428e-b6b8-d7fae95ef3b2" containerID="2e2cc4c4daf44dd55b2740b3ea8aad6e1392ef147914ce9a003c424ff30b01ff" exitCode=0
Jul 06 18:31:34.876304 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:31:34.876170    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-c84d79b86-v95hp"
Jul 06 18:31:34.876304 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:31:34.876193    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-c84d79b86-v95hp" event={"ID":"9ab48cb5-c3b3-428e-b6b8-d7fae95ef3b2","Type":"ContainerDied","Data":"2e2cc4c4daf44dd55b2740b3ea8aad6e1392ef147914ce9a003c424ff30b01ff"}
Jul 06 18:31:34.876304 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:31:34.876234    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-c84d79b86-v95hp" event={"ID":"9ab48cb5-c3b3-428e-b6b8-d7fae95ef3b2","Type":"ContainerDied","Data":"2107f07fcd9b9fbf6d8bbfc6732977411b3c9a112e54aa563919717065c7ea00"}
Jul 06 18:31:34.876304 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:31:34.876254    2580 scope.go:117] "RemoveContainer" containerID="2e2cc4c4daf44dd55b2740b3ea8aad6e1392ef147914ce9a003c424ff30b01ff"
Jul 06 18:31:34.891628 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:31:34.891605    2580 scope.go:117] "RemoveContainer" containerID="2e2cc4c4daf44dd55b2740b3ea8aad6e1392ef147914ce9a003c424ff30b01ff"
Jul 06 18:31:34.891934 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:31:34.891912    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2e2cc4c4daf44dd55b2740b3ea8aad6e1392ef147914ce9a003c424ff30b01ff\": container with ID starting with 2e2cc4c4daf44dd55b2740b3ea8aad6e1392ef147914ce9a003c424ff30b01ff not found: ID does not exist" containerID="2e2cc4c4daf44dd55b2740b3ea8aad6e1392ef147914ce9a003c424ff30b01ff"
Jul 06 18:31:34.892053 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:31:34.891945    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2e2cc4c4daf44dd55b2740b3ea8aad6e1392ef147914ce9a003c424ff30b01ff"} err="failed to get container status \"2e2cc4c4daf44dd55b2740b3ea8aad6e1392ef147914ce9a003c424ff30b01ff\": rpc error: code = NotFound desc = could not find container \"2e2cc4c4daf44dd55b2740b3ea8aad6e1392ef147914ce9a003c424ff30b01ff\": container with ID starting with 2e2cc4c4daf44dd55b2740b3ea8aad6e1392ef147914ce9a003c424ff30b01ff not found: ID does not exist"
Jul 06 18:31:34.902241 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:31:34.902215    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-c84d79b86-v95hp"]
Jul 06 18:31:34.905556 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:31:34.905532    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-c84d79b86-v95hp"]
Jul 06 18:31:35.322872 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:31:35.322846    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9ab48cb5-c3b3-428e-b6b8-d7fae95ef3b2" path="/var/lib/kubelet/pods/9ab48cb5-c3b3-428e-b6b8-d7fae95ef3b2/volumes"
Jul 06 18:31:35.703547 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:31:35.703470    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-c84d79b86-pc4x6"]
Jul 06 18:31:35.703845 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:31:35.703832    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ddfe9a5b-4ef3-4bad-8c95-9ab633bf51a2" containerName="authorino"
Jul 06 18:31:35.703885 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:31:35.703847    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="ddfe9a5b-4ef3-4bad-8c95-9ab633bf51a2" containerName="authorino"
Jul 06 18:31:35.703885 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:31:35.703873    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ab48cb5-c3b3-428e-b6b8-d7fae95ef3b2" containerName="manager"
Jul 06 18:31:35.703885 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:31:35.703879    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ab48cb5-c3b3-428e-b6b8-d7fae95ef3b2" containerName="manager"
Jul 06 18:31:35.703978 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:31:35.703933    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="ddfe9a5b-4ef3-4bad-8c95-9ab633bf51a2" containerName="authorino"
Jul 06 18:31:35.703978 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:31:35.703944    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="9ab48cb5-c3b3-428e-b6b8-d7fae95ef3b2" containerName="manager"
Jul 06 18:31:35.708430 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:31:35.708411    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-c84d79b86-pc4x6"
Jul 06 18:31:35.712286 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:31:35.712263    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-cxmll\""
Jul 06 18:31:35.712455 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:31:35.712274    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 06 18:31:35.712455 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:31:35.712309    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 06 18:31:35.715923 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:31:35.715899    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-c84d79b86-pc4x6"]
Jul 06 18:31:35.789396 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:31:35.789359    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kgtzz\" (UniqueName: \"kubernetes.io/projected/ab01db64-3934-426a-a989-760248f8c0cd-kube-api-access-kgtzz\") pod \"maas-controller-c84d79b86-pc4x6\" (UID: \"ab01db64-3934-426a-a989-760248f8c0cd\") " pod="opendatahub/maas-controller-c84d79b86-pc4x6"
Jul 06 18:31:35.789579 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:31:35.789421    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ab01db64-3934-426a-a989-760248f8c0cd-webhook-cert\") pod \"maas-controller-c84d79b86-pc4x6\" (UID: \"ab01db64-3934-426a-a989-760248f8c0cd\") " pod="opendatahub/maas-controller-c84d79b86-pc4x6"
Jul 06 18:31:35.890464 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:31:35.890427    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ab01db64-3934-426a-a989-760248f8c0cd-webhook-cert\") pod \"maas-controller-c84d79b86-pc4x6\" (UID: \"ab01db64-3934-426a-a989-760248f8c0cd\") " pod="opendatahub/maas-controller-c84d79b86-pc4x6"
Jul 06 18:31:35.890668 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:31:35.890506    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kgtzz\" (UniqueName: \"kubernetes.io/projected/ab01db64-3934-426a-a989-760248f8c0cd-kube-api-access-kgtzz\") pod \"maas-controller-c84d79b86-pc4x6\" (UID: \"ab01db64-3934-426a-a989-760248f8c0cd\") " pod="opendatahub/maas-controller-c84d79b86-pc4x6"
Jul 06 18:31:35.893029 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:31:35.892985    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ab01db64-3934-426a-a989-760248f8c0cd-webhook-cert\") pod \"maas-controller-c84d79b86-pc4x6\" (UID: \"ab01db64-3934-426a-a989-760248f8c0cd\") " pod="opendatahub/maas-controller-c84d79b86-pc4x6"
Jul 06 18:31:35.899454 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:31:35.899405    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kgtzz\" (UniqueName: \"kubernetes.io/projected/ab01db64-3934-426a-a989-760248f8c0cd-kube-api-access-kgtzz\") pod \"maas-controller-c84d79b86-pc4x6\" (UID: \"ab01db64-3934-426a-a989-760248f8c0cd\") " pod="opendatahub/maas-controller-c84d79b86-pc4x6"
Jul 06 18:31:36.019891 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:31:36.019805    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-c84d79b86-pc4x6"
Jul 06 18:31:36.150346 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:31:36.150314    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-c84d79b86-pc4x6"]
Jul 06 18:31:36.153681 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:31:36.153654    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab01db64_3934_426a_a989_760248f8c0cd.slice/crio-7417dde53804a7d56752f6e84ad1c08fffdc24c89c29313e9fca286317fa4bb3 WatchSource:0}: Error finding container 7417dde53804a7d56752f6e84ad1c08fffdc24c89c29313e9fca286317fa4bb3: Status 404 returned error can't find the container with id 7417dde53804a7d56752f6e84ad1c08fffdc24c89c29313e9fca286317fa4bb3
Jul 06 18:31:36.888470 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:31:36.888428    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-c84d79b86-pc4x6" event={"ID":"ab01db64-3934-426a-a989-760248f8c0cd","Type":"ContainerStarted","Data":"d98fd0d57dd40480ffb71029fcae886501e696bf1064bbb2cab1dc9859b1a6c2"}
Jul 06 18:31:36.888862 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:31:36.888490    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-c84d79b86-pc4x6" event={"ID":"ab01db64-3934-426a-a989-760248f8c0cd","Type":"ContainerStarted","Data":"7417dde53804a7d56752f6e84ad1c08fffdc24c89c29313e9fca286317fa4bb3"}
Jul 06 18:31:36.888862 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:31:36.888610    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-c84d79b86-pc4x6"
Jul 06 18:31:36.905053 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:31:36.904984    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-c84d79b86-pc4x6" podStartSLOduration=1.302950216 podStartE2EDuration="1.904969189s" totalImagesPullingTime="602.018973ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:31:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:31:36.902609308 +0000 UTC m=+832.060007797" watchObservedRunningTime="2026-07-06 18:31:36.904969189 +0000 UTC m=+832.062367694"
Jul 06 18:31:47.898465 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:31:47.898435    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-c84d79b86-pc4x6"
Jul 06 18:43:31.624999 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:31.624900    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-daa9ff09-openshift-default-996c88c-nxzb5"]
Jul 06 18:43:31.628591 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:31.628566    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-daa9ff09-openshift-default-996c88c-nxzb5"
Jul 06 18:43:31.631584 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:31.631554    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-daa9ff09-openshift-default-dockercfg-4qwzd\""
Jul 06 18:43:31.651645 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:31.651611    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-daa9ff09-openshift-default-996c88c-nxzb5"]
Jul 06 18:43:31.699896 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:31.699859    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/65f72f88-3c37-4780-adf0-de18fcff10f8-istio-data\") pod \"e2e-ait-daa9ff09-openshift-default-996c88c-nxzb5\" (UID: \"65f72f88-3c37-4780-adf0-de18fcff10f8\") " pod="openshift-ingress/e2e-ait-daa9ff09-openshift-default-996c88c-nxzb5"
Jul 06 18:43:31.699896 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:31.699900    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/65f72f88-3c37-4780-adf0-de18fcff10f8-istio-token\") pod \"e2e-ait-daa9ff09-openshift-default-996c88c-nxzb5\" (UID: \"65f72f88-3c37-4780-adf0-de18fcff10f8\") " pod="openshift-ingress/e2e-ait-daa9ff09-openshift-default-996c88c-nxzb5"
Jul 06 18:43:31.700134 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:31.699979    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/65f72f88-3c37-4780-adf0-de18fcff10f8-workload-socket\") pod \"e2e-ait-daa9ff09-openshift-default-996c88c-nxzb5\" (UID: \"65f72f88-3c37-4780-adf0-de18fcff10f8\") " pod="openshift-ingress/e2e-ait-daa9ff09-openshift-default-996c88c-nxzb5"
Jul 06 18:43:31.700134 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:31.700051    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/65f72f88-3c37-4780-adf0-de18fcff10f8-istio-envoy\") pod \"e2e-ait-daa9ff09-openshift-default-996c88c-nxzb5\" (UID: \"65f72f88-3c37-4780-adf0-de18fcff10f8\") " pod="openshift-ingress/e2e-ait-daa9ff09-openshift-default-996c88c-nxzb5"
Jul 06 18:43:31.700134 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:31.700075    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/65f72f88-3c37-4780-adf0-de18fcff10f8-istio-podinfo\") pod \"e2e-ait-daa9ff09-openshift-default-996c88c-nxzb5\" (UID: \"65f72f88-3c37-4780-adf0-de18fcff10f8\") " pod="openshift-ingress/e2e-ait-daa9ff09-openshift-default-996c88c-nxzb5"
Jul 06 18:43:31.700134 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:31.700096    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/65f72f88-3c37-4780-adf0-de18fcff10f8-istiod-ca-cert\") pod \"e2e-ait-daa9ff09-openshift-default-996c88c-nxzb5\" (UID: \"65f72f88-3c37-4780-adf0-de18fcff10f8\") " pod="openshift-ingress/e2e-ait-daa9ff09-openshift-default-996c88c-nxzb5"
Jul 06 18:43:31.700281 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:31.700139    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/65f72f88-3c37-4780-adf0-de18fcff10f8-credential-socket\") pod \"e2e-ait-daa9ff09-openshift-default-996c88c-nxzb5\" (UID: \"65f72f88-3c37-4780-adf0-de18fcff10f8\") " pod="openshift-ingress/e2e-ait-daa9ff09-openshift-default-996c88c-nxzb5"
Jul 06 18:43:31.700281 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:31.700165    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/65f72f88-3c37-4780-adf0-de18fcff10f8-workload-certs\") pod \"e2e-ait-daa9ff09-openshift-default-996c88c-nxzb5\" (UID: \"65f72f88-3c37-4780-adf0-de18fcff10f8\") " pod="openshift-ingress/e2e-ait-daa9ff09-openshift-default-996c88c-nxzb5"
Jul 06 18:43:31.700281 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:31.700189    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n8wvc\" (UniqueName: \"kubernetes.io/projected/65f72f88-3c37-4780-adf0-de18fcff10f8-kube-api-access-n8wvc\") pod \"e2e-ait-daa9ff09-openshift-default-996c88c-nxzb5\" (UID: \"65f72f88-3c37-4780-adf0-de18fcff10f8\") " pod="openshift-ingress/e2e-ait-daa9ff09-openshift-default-996c88c-nxzb5"
Jul 06 18:43:31.801268 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:31.801224    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/65f72f88-3c37-4780-adf0-de18fcff10f8-credential-socket\") pod \"e2e-ait-daa9ff09-openshift-default-996c88c-nxzb5\" (UID: \"65f72f88-3c37-4780-adf0-de18fcff10f8\") " pod="openshift-ingress/e2e-ait-daa9ff09-openshift-default-996c88c-nxzb5"
Jul 06 18:43:31.801456 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:31.801277    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/65f72f88-3c37-4780-adf0-de18fcff10f8-workload-certs\") pod \"e2e-ait-daa9ff09-openshift-default-996c88c-nxzb5\" (UID: \"65f72f88-3c37-4780-adf0-de18fcff10f8\") " pod="openshift-ingress/e2e-ait-daa9ff09-openshift-default-996c88c-nxzb5"
Jul 06 18:43:31.801456 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:31.801312    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n8wvc\" (UniqueName: \"kubernetes.io/projected/65f72f88-3c37-4780-adf0-de18fcff10f8-kube-api-access-n8wvc\") pod \"e2e-ait-daa9ff09-openshift-default-996c88c-nxzb5\" (UID: \"65f72f88-3c37-4780-adf0-de18fcff10f8\") " pod="openshift-ingress/e2e-ait-daa9ff09-openshift-default-996c88c-nxzb5"
Jul 06 18:43:31.801456 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:31.801360    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/65f72f88-3c37-4780-adf0-de18fcff10f8-istio-data\") pod \"e2e-ait-daa9ff09-openshift-default-996c88c-nxzb5\" (UID: \"65f72f88-3c37-4780-adf0-de18fcff10f8\") " pod="openshift-ingress/e2e-ait-daa9ff09-openshift-default-996c88c-nxzb5"
Jul 06 18:43:31.801456 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:31.801387    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/65f72f88-3c37-4780-adf0-de18fcff10f8-istio-token\") pod \"e2e-ait-daa9ff09-openshift-default-996c88c-nxzb5\" (UID: \"65f72f88-3c37-4780-adf0-de18fcff10f8\") " pod="openshift-ingress/e2e-ait-daa9ff09-openshift-default-996c88c-nxzb5"
Jul 06 18:43:31.801456 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:31.801439    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/65f72f88-3c37-4780-adf0-de18fcff10f8-workload-socket\") pod \"e2e-ait-daa9ff09-openshift-default-996c88c-nxzb5\" (UID: \"65f72f88-3c37-4780-adf0-de18fcff10f8\") " pod="openshift-ingress/e2e-ait-daa9ff09-openshift-default-996c88c-nxzb5"
Jul 06 18:43:31.801713 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:31.801502    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/65f72f88-3c37-4780-adf0-de18fcff10f8-istio-envoy\") pod \"e2e-ait-daa9ff09-openshift-default-996c88c-nxzb5\" (UID: \"65f72f88-3c37-4780-adf0-de18fcff10f8\") " pod="openshift-ingress/e2e-ait-daa9ff09-openshift-default-996c88c-nxzb5"
Jul 06 18:43:31.801713 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:31.801536    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/65f72f88-3c37-4780-adf0-de18fcff10f8-istio-podinfo\") pod \"e2e-ait-daa9ff09-openshift-default-996c88c-nxzb5\" (UID: \"65f72f88-3c37-4780-adf0-de18fcff10f8\") " pod="openshift-ingress/e2e-ait-daa9ff09-openshift-default-996c88c-nxzb5"
Jul 06 18:43:31.801713 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:31.801565    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/65f72f88-3c37-4780-adf0-de18fcff10f8-istiod-ca-cert\") pod \"e2e-ait-daa9ff09-openshift-default-996c88c-nxzb5\" (UID: \"65f72f88-3c37-4780-adf0-de18fcff10f8\") " pod="openshift-ingress/e2e-ait-daa9ff09-openshift-default-996c88c-nxzb5"
Jul 06 18:43:31.801869 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:31.801724    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/65f72f88-3c37-4780-adf0-de18fcff10f8-credential-socket\") pod \"e2e-ait-daa9ff09-openshift-default-996c88c-nxzb5\" (UID: \"65f72f88-3c37-4780-adf0-de18fcff10f8\") " pod="openshift-ingress/e2e-ait-daa9ff09-openshift-default-996c88c-nxzb5"
Jul 06 18:43:31.801869 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:31.801823    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/65f72f88-3c37-4780-adf0-de18fcff10f8-istio-data\") pod \"e2e-ait-daa9ff09-openshift-default-996c88c-nxzb5\" (UID: \"65f72f88-3c37-4780-adf0-de18fcff10f8\") " pod="openshift-ingress/e2e-ait-daa9ff09-openshift-default-996c88c-nxzb5"
Jul 06 18:43:31.801869 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:31.801842    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/65f72f88-3c37-4780-adf0-de18fcff10f8-workload-certs\") pod \"e2e-ait-daa9ff09-openshift-default-996c88c-nxzb5\" (UID: \"65f72f88-3c37-4780-adf0-de18fcff10f8\") " pod="openshift-ingress/e2e-ait-daa9ff09-openshift-default-996c88c-nxzb5"
Jul 06 18:43:31.801968 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:31.801938    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/65f72f88-3c37-4780-adf0-de18fcff10f8-workload-socket\") pod \"e2e-ait-daa9ff09-openshift-default-996c88c-nxzb5\" (UID: \"65f72f88-3c37-4780-adf0-de18fcff10f8\") " pod="openshift-ingress/e2e-ait-daa9ff09-openshift-default-996c88c-nxzb5"
Jul 06 18:43:31.802570 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:31.802549    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/65f72f88-3c37-4780-adf0-de18fcff10f8-istiod-ca-cert\") pod \"e2e-ait-daa9ff09-openshift-default-996c88c-nxzb5\" (UID: \"65f72f88-3c37-4780-adf0-de18fcff10f8\") " pod="openshift-ingress/e2e-ait-daa9ff09-openshift-default-996c88c-nxzb5"
Jul 06 18:43:31.803807 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:31.803789    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/65f72f88-3c37-4780-adf0-de18fcff10f8-istio-envoy\") pod \"e2e-ait-daa9ff09-openshift-default-996c88c-nxzb5\" (UID: \"65f72f88-3c37-4780-adf0-de18fcff10f8\") " pod="openshift-ingress/e2e-ait-daa9ff09-openshift-default-996c88c-nxzb5"
Jul 06 18:43:31.804073 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:31.804050    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/65f72f88-3c37-4780-adf0-de18fcff10f8-istio-podinfo\") pod \"e2e-ait-daa9ff09-openshift-default-996c88c-nxzb5\" (UID: \"65f72f88-3c37-4780-adf0-de18fcff10f8\") " pod="openshift-ingress/e2e-ait-daa9ff09-openshift-default-996c88c-nxzb5"
Jul 06 18:43:31.810240 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:31.810216    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/65f72f88-3c37-4780-adf0-de18fcff10f8-istio-token\") pod \"e2e-ait-daa9ff09-openshift-default-996c88c-nxzb5\" (UID: \"65f72f88-3c37-4780-adf0-de18fcff10f8\") " pod="openshift-ingress/e2e-ait-daa9ff09-openshift-default-996c88c-nxzb5"
Jul 06 18:43:31.810370 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:31.810351    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n8wvc\" (UniqueName: \"kubernetes.io/projected/65f72f88-3c37-4780-adf0-de18fcff10f8-kube-api-access-n8wvc\") pod \"e2e-ait-daa9ff09-openshift-default-996c88c-nxzb5\" (UID: \"65f72f88-3c37-4780-adf0-de18fcff10f8\") " pod="openshift-ingress/e2e-ait-daa9ff09-openshift-default-996c88c-nxzb5"
Jul 06 18:43:31.939692 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:31.939613    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-daa9ff09-openshift-default-996c88c-nxzb5"
Jul 06 18:43:32.079278 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:32.079253    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-daa9ff09-openshift-default-996c88c-nxzb5"]
Jul 06 18:43:32.081180 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:43:32.081151    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65f72f88_3c37_4780_adf0_de18fcff10f8.slice/crio-57c9a0733c8724a355d2060cb20da936670da32c48bb933d6e1623a0a04dd19b WatchSource:0}: Error finding container 57c9a0733c8724a355d2060cb20da936670da32c48bb933d6e1623a0a04dd19b: Status 404 returned error can't find the container with id 57c9a0733c8724a355d2060cb20da936670da32c48bb933d6e1623a0a04dd19b
Jul 06 18:43:32.083155 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:32.083124    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 06 18:43:32.083248 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:32.083199    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 06 18:43:32.083305 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:32.083277    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 06 18:43:32.172907 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:32.172872    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-daa9ff09-openshift-default-996c88c-nxzb5" event={"ID":"65f72f88-3c37-4780-adf0-de18fcff10f8","Type":"ContainerStarted","Data":"36682c0702bd8efc2928ed5949c50adc5f37473688c7ced52db08e316c91d08a"}
Jul 06 18:43:32.173084 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:32.172930    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-daa9ff09-openshift-default-996c88c-nxzb5" event={"ID":"65f72f88-3c37-4780-adf0-de18fcff10f8","Type":"ContainerStarted","Data":"57c9a0733c8724a355d2060cb20da936670da32c48bb933d6e1623a0a04dd19b"}
Jul 06 18:43:32.195226 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:32.195121    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-daa9ff09-openshift-default-996c88c-nxzb5" podStartSLOduration=1.195105437 podStartE2EDuration="1.195105437s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:43:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:43:32.193976612 +0000 UTC m=+1547.351375109" watchObservedRunningTime="2026-07-06 18:43:32.195105437 +0000 UTC m=+1547.352503925"
Jul 06 18:43:32.917412 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:32.917371    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-ait-daa9ff09-9b7fd6fb5-fknpz"]
Jul 06 18:43:32.921701 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:32.921679    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-daa9ff09-9b7fd6fb5-fknpz"
Jul 06 18:43:32.924202 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:32.924178    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-ait-daa9ff09-serving-cert\""
Jul 06 18:43:32.931903 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:32.931880    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-daa9ff09-9b7fd6fb5-fknpz"]
Jul 06 18:43:32.940374 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:32.940352    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-daa9ff09-openshift-default-996c88c-nxzb5"
Jul 06 18:43:33.013299 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:33.013256    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sx9fr\" (UniqueName: \"kubernetes.io/projected/4e9f4b6d-8898-41c6-b490-7328e50223d3-kube-api-access-sx9fr\") pod \"maas-api-e2e-ait-daa9ff09-9b7fd6fb5-fknpz\" (UID: \"4e9f4b6d-8898-41c6-b490-7328e50223d3\") " pod="opendatahub/maas-api-e2e-ait-daa9ff09-9b7fd6fb5-fknpz"
Jul 06 18:43:33.013480 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:33.013313    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e9f4b6d-8898-41c6-b490-7328e50223d3-openshift-service-ca\") pod \"maas-api-e2e-ait-daa9ff09-9b7fd6fb5-fknpz\" (UID: \"4e9f4b6d-8898-41c6-b490-7328e50223d3\") " pod="opendatahub/maas-api-e2e-ait-daa9ff09-9b7fd6fb5-fknpz"
Jul 06 18:43:33.013480 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:33.013368    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4e9f4b6d-8898-41c6-b490-7328e50223d3-maas-api-tls\") pod \"maas-api-e2e-ait-daa9ff09-9b7fd6fb5-fknpz\" (UID: \"4e9f4b6d-8898-41c6-b490-7328e50223d3\") " pod="opendatahub/maas-api-e2e-ait-daa9ff09-9b7fd6fb5-fknpz"
Jul 06 18:43:33.114400 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:33.114365    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4e9f4b6d-8898-41c6-b490-7328e50223d3-maas-api-tls\") pod \"maas-api-e2e-ait-daa9ff09-9b7fd6fb5-fknpz\" (UID: \"4e9f4b6d-8898-41c6-b490-7328e50223d3\") " pod="opendatahub/maas-api-e2e-ait-daa9ff09-9b7fd6fb5-fknpz"
Jul 06 18:43:33.114591 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:33.114534    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sx9fr\" (UniqueName: \"kubernetes.io/projected/4e9f4b6d-8898-41c6-b490-7328e50223d3-kube-api-access-sx9fr\") pod \"maas-api-e2e-ait-daa9ff09-9b7fd6fb5-fknpz\" (UID: \"4e9f4b6d-8898-41c6-b490-7328e50223d3\") " pod="opendatahub/maas-api-e2e-ait-daa9ff09-9b7fd6fb5-fknpz"
Jul 06 18:43:33.114664 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:33.114612    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e9f4b6d-8898-41c6-b490-7328e50223d3-openshift-service-ca\") pod \"maas-api-e2e-ait-daa9ff09-9b7fd6fb5-fknpz\" (UID: \"4e9f4b6d-8898-41c6-b490-7328e50223d3\") " pod="opendatahub/maas-api-e2e-ait-daa9ff09-9b7fd6fb5-fknpz"
Jul 06 18:43:33.115303 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:33.115276    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e9f4b6d-8898-41c6-b490-7328e50223d3-openshift-service-ca\") pod \"maas-api-e2e-ait-daa9ff09-9b7fd6fb5-fknpz\" (UID: \"4e9f4b6d-8898-41c6-b490-7328e50223d3\") " pod="opendatahub/maas-api-e2e-ait-daa9ff09-9b7fd6fb5-fknpz"
Jul 06 18:43:33.116887 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:33.116863    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4e9f4b6d-8898-41c6-b490-7328e50223d3-maas-api-tls\") pod \"maas-api-e2e-ait-daa9ff09-9b7fd6fb5-fknpz\" (UID: \"4e9f4b6d-8898-41c6-b490-7328e50223d3\") " pod="opendatahub/maas-api-e2e-ait-daa9ff09-9b7fd6fb5-fknpz"
Jul 06 18:43:33.126178 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:33.126153    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sx9fr\" (UniqueName: \"kubernetes.io/projected/4e9f4b6d-8898-41c6-b490-7328e50223d3-kube-api-access-sx9fr\") pod \"maas-api-e2e-ait-daa9ff09-9b7fd6fb5-fknpz\" (UID: \"4e9f4b6d-8898-41c6-b490-7328e50223d3\") " pod="opendatahub/maas-api-e2e-ait-daa9ff09-9b7fd6fb5-fknpz"
Jul 06 18:43:33.236658 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:33.236568    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-daa9ff09-9b7fd6fb5-fknpz"
Jul 06 18:43:33.379964 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:33.379936    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-daa9ff09-9b7fd6fb5-fknpz"]
Jul 06 18:43:33.384785 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:33.384755    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 06 18:43:33.945493 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:33.945466    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-daa9ff09-openshift-default-996c88c-nxzb5"
Jul 06 18:43:34.182710 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:34.182676    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-daa9ff09-9b7fd6fb5-fknpz" event={"ID":"4e9f4b6d-8898-41c6-b490-7328e50223d3","Type":"ContainerStarted","Data":"061487a8682842aa3c76f8ebf62be2edce4de71cdeb4930cd9da40b4a07da847"}
Jul 06 18:43:34.182878 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:34.182747    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-daa9ff09-openshift-default-996c88c-nxzb5"
Jul 06 18:43:34.182878 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:34.182771    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-daa9ff09-9b7fd6fb5-fknpz" event={"ID":"4e9f4b6d-8898-41c6-b490-7328e50223d3","Type":"ContainerStarted","Data":"f8c22809de5f74a7d2fcdd7bdca1a68fbbe78014f60773b083c6ed434b03a92a"}
Jul 06 18:43:34.183003 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:34.182888    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-ait-daa9ff09-9b7fd6fb5-fknpz"
Jul 06 18:43:34.183779 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:34.183765    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-daa9ff09-openshift-default-996c88c-nxzb5"
Jul 06 18:43:34.202626 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:34.202545    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-ait-daa9ff09-9b7fd6fb5-fknpz" podStartSLOduration=1.613334407 podStartE2EDuration="2.202532631s" totalImagesPullingTime="589.198224ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:43:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:43:34.20105289 +0000 UTC m=+1549.358451377" watchObservedRunningTime="2026-07-06 18:43:34.202532631 +0000 UTC m=+1549.359931118"
Jul 06 18:43:38.134239 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:38.134178    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-daa9ff09-openshift-default-996c88c-nxzb5"]
Jul 06 18:43:38.134909 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:38.134489    2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-daa9ff09-openshift-default-996c88c-nxzb5" podUID="65f72f88-3c37-4780-adf0-de18fcff10f8" containerName="istio-proxy" containerID="cri-o://36682c0702bd8efc2928ed5949c50adc5f37473688c7ced52db08e316c91d08a" gracePeriod=30
Jul 06 18:43:38.443169 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:38.443063    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-daa9ff09-9b7fd6fb5-fknpz"]
Jul 06 18:43:38.443625 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:38.443590    2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-ait-daa9ff09-9b7fd6fb5-fknpz" podUID="4e9f4b6d-8898-41c6-b490-7328e50223d3" containerName="maas-api" containerID="cri-o://061487a8682842aa3c76f8ebf62be2edce4de71cdeb4930cd9da40b4a07da847" gracePeriod=30
Jul 06 18:43:38.719701 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:38.719675    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-daa9ff09-9b7fd6fb5-fknpz"
Jul 06 18:43:38.768989 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:38.768946    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4e9f4b6d-8898-41c6-b490-7328e50223d3-maas-api-tls\") pod \"4e9f4b6d-8898-41c6-b490-7328e50223d3\" (UID: \"4e9f4b6d-8898-41c6-b490-7328e50223d3\") "
Jul 06 18:43:38.769174 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:38.769028    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e9f4b6d-8898-41c6-b490-7328e50223d3-openshift-service-ca\") pod \"4e9f4b6d-8898-41c6-b490-7328e50223d3\" (UID: \"4e9f4b6d-8898-41c6-b490-7328e50223d3\") "
Jul 06 18:43:38.769174 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:38.769136    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sx9fr\" (UniqueName: \"kubernetes.io/projected/4e9f4b6d-8898-41c6-b490-7328e50223d3-kube-api-access-sx9fr\") pod \"4e9f4b6d-8898-41c6-b490-7328e50223d3\" (UID: \"4e9f4b6d-8898-41c6-b490-7328e50223d3\") "
Jul 06 18:43:38.769329 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:38.769300    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4e9f4b6d-8898-41c6-b490-7328e50223d3-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "4e9f4b6d-8898-41c6-b490-7328e50223d3" (UID: "4e9f4b6d-8898-41c6-b490-7328e50223d3"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 18:43:38.769440 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:38.769411    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e9f4b6d-8898-41c6-b490-7328e50223d3-openshift-service-ca\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:43:38.771990 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:38.771964    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4e9f4b6d-8898-41c6-b490-7328e50223d3-kube-api-access-sx9fr" (OuterVolumeSpecName: "kube-api-access-sx9fr") pod "4e9f4b6d-8898-41c6-b490-7328e50223d3" (UID: "4e9f4b6d-8898-41c6-b490-7328e50223d3"). InnerVolumeSpecName "kube-api-access-sx9fr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 18:43:38.772494 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:38.772471    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4e9f4b6d-8898-41c6-b490-7328e50223d3-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "4e9f4b6d-8898-41c6-b490-7328e50223d3" (UID: "4e9f4b6d-8898-41c6-b490-7328e50223d3"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 18:43:38.870715 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:38.870676    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sx9fr\" (UniqueName: \"kubernetes.io/projected/4e9f4b6d-8898-41c6-b490-7328e50223d3-kube-api-access-sx9fr\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:43:38.870715 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:38.870706    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4e9f4b6d-8898-41c6-b490-7328e50223d3-maas-api-tls\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:43:39.208820 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:39.208783    2580 generic.go:356] "Generic (PLEG): container finished" podID="4e9f4b6d-8898-41c6-b490-7328e50223d3" containerID="061487a8682842aa3c76f8ebf62be2edce4de71cdeb4930cd9da40b4a07da847" exitCode=0
Jul 06 18:43:39.209372 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:39.208831    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-daa9ff09-9b7fd6fb5-fknpz"
Jul 06 18:43:39.209372 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:39.208869    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-daa9ff09-9b7fd6fb5-fknpz" event={"ID":"4e9f4b6d-8898-41c6-b490-7328e50223d3","Type":"ContainerDied","Data":"061487a8682842aa3c76f8ebf62be2edce4de71cdeb4930cd9da40b4a07da847"}
Jul 06 18:43:39.209372 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:39.208931    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-daa9ff09-9b7fd6fb5-fknpz" event={"ID":"4e9f4b6d-8898-41c6-b490-7328e50223d3","Type":"ContainerDied","Data":"f8c22809de5f74a7d2fcdd7bdca1a68fbbe78014f60773b083c6ed434b03a92a"}
Jul 06 18:43:39.209372 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:39.208964    2580 scope.go:117] "RemoveContainer" containerID="061487a8682842aa3c76f8ebf62be2edce4de71cdeb4930cd9da40b4a07da847"
Jul 06 18:43:39.218376 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:39.218361    2580 scope.go:117] "RemoveContainer" containerID="061487a8682842aa3c76f8ebf62be2edce4de71cdeb4930cd9da40b4a07da847"
Jul 06 18:43:39.218627 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:43:39.218604    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"061487a8682842aa3c76f8ebf62be2edce4de71cdeb4930cd9da40b4a07da847\": container with ID starting with 061487a8682842aa3c76f8ebf62be2edce4de71cdeb4930cd9da40b4a07da847 not found: ID does not exist" containerID="061487a8682842aa3c76f8ebf62be2edce4de71cdeb4930cd9da40b4a07da847"
Jul 06 18:43:39.218670 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:39.218635    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"061487a8682842aa3c76f8ebf62be2edce4de71cdeb4930cd9da40b4a07da847"} err="failed to get container status \"061487a8682842aa3c76f8ebf62be2edce4de71cdeb4930cd9da40b4a07da847\": rpc error: code = NotFound desc = could not find container \"061487a8682842aa3c76f8ebf62be2edce4de71cdeb4930cd9da40b4a07da847\": container with ID starting with 061487a8682842aa3c76f8ebf62be2edce4de71cdeb4930cd9da40b4a07da847 not found: ID does not exist"
Jul 06 18:43:39.231475 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:39.231439    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-daa9ff09-9b7fd6fb5-fknpz"]
Jul 06 18:43:39.235533 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:39.235507    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-ait-daa9ff09-9b7fd6fb5-fknpz"]
Jul 06 18:43:39.323403 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:39.323367    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4e9f4b6d-8898-41c6-b490-7328e50223d3" path="/var/lib/kubelet/pods/4e9f4b6d-8898-41c6-b490-7328e50223d3/volumes"
Jul 06 18:43:43.991179 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:43.991121    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-daa9ff09-openshift-default-996c88c-nxzb5"
Jul 06 18:43:44.120768 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:44.120729    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/65f72f88-3c37-4780-adf0-de18fcff10f8-istio-token\") pod \"65f72f88-3c37-4780-adf0-de18fcff10f8\" (UID: \"65f72f88-3c37-4780-adf0-de18fcff10f8\") "
Jul 06 18:43:44.120768 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:44.120771    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/65f72f88-3c37-4780-adf0-de18fcff10f8-workload-certs\") pod \"65f72f88-3c37-4780-adf0-de18fcff10f8\" (UID: \"65f72f88-3c37-4780-adf0-de18fcff10f8\") "
Jul 06 18:43:44.120985 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:44.120801    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/65f72f88-3c37-4780-adf0-de18fcff10f8-istio-data\") pod \"65f72f88-3c37-4780-adf0-de18fcff10f8\" (UID: \"65f72f88-3c37-4780-adf0-de18fcff10f8\") "
Jul 06 18:43:44.120985 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:44.120842    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/65f72f88-3c37-4780-adf0-de18fcff10f8-istio-podinfo\") pod \"65f72f88-3c37-4780-adf0-de18fcff10f8\" (UID: \"65f72f88-3c37-4780-adf0-de18fcff10f8\") "
Jul 06 18:43:44.120985 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:44.120872    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/65f72f88-3c37-4780-adf0-de18fcff10f8-istiod-ca-cert\") pod \"65f72f88-3c37-4780-adf0-de18fcff10f8\" (UID: \"65f72f88-3c37-4780-adf0-de18fcff10f8\") "
Jul 06 18:43:44.120985 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:44.120896    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n8wvc\" (UniqueName: \"kubernetes.io/projected/65f72f88-3c37-4780-adf0-de18fcff10f8-kube-api-access-n8wvc\") pod \"65f72f88-3c37-4780-adf0-de18fcff10f8\" (UID: \"65f72f88-3c37-4780-adf0-de18fcff10f8\") "
Jul 06 18:43:44.120985 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:44.120921    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/65f72f88-3c37-4780-adf0-de18fcff10f8-istio-envoy\") pod \"65f72f88-3c37-4780-adf0-de18fcff10f8\" (UID: \"65f72f88-3c37-4780-adf0-de18fcff10f8\") "
Jul 06 18:43:44.120985 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:44.120951    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/65f72f88-3c37-4780-adf0-de18fcff10f8-workload-socket\") pod \"65f72f88-3c37-4780-adf0-de18fcff10f8\" (UID: \"65f72f88-3c37-4780-adf0-de18fcff10f8\") "
Jul 06 18:43:44.120985 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:44.120976    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/65f72f88-3c37-4780-adf0-de18fcff10f8-credential-socket\") pod \"65f72f88-3c37-4780-adf0-de18fcff10f8\" (UID: \"65f72f88-3c37-4780-adf0-de18fcff10f8\") "
Jul 06 18:43:44.121356 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:44.121066    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/65f72f88-3c37-4780-adf0-de18fcff10f8-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "65f72f88-3c37-4780-adf0-de18fcff10f8" (UID: "65f72f88-3c37-4780-adf0-de18fcff10f8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 18:43:44.121356 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:44.121259    2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/65f72f88-3c37-4780-adf0-de18fcff10f8-workload-certs\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:43:44.121356 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:44.121287    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/65f72f88-3c37-4780-adf0-de18fcff10f8-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "65f72f88-3c37-4780-adf0-de18fcff10f8" (UID: "65f72f88-3c37-4780-adf0-de18fcff10f8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 18:43:44.121513 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:44.121403    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/65f72f88-3c37-4780-adf0-de18fcff10f8-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "65f72f88-3c37-4780-adf0-de18fcff10f8" (UID: "65f72f88-3c37-4780-adf0-de18fcff10f8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 18:43:44.121624 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:44.121598    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/65f72f88-3c37-4780-adf0-de18fcff10f8-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "65f72f88-3c37-4780-adf0-de18fcff10f8" (UID: "65f72f88-3c37-4780-adf0-de18fcff10f8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 18:43:44.121857 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:44.121832    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/65f72f88-3c37-4780-adf0-de18fcff10f8-istio-data" (OuterVolumeSpecName: "istio-data") pod "65f72f88-3c37-4780-adf0-de18fcff10f8" (UID: "65f72f88-3c37-4780-adf0-de18fcff10f8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 18:43:44.123137 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:44.123116    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/65f72f88-3c37-4780-adf0-de18fcff10f8-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "65f72f88-3c37-4780-adf0-de18fcff10f8" (UID: "65f72f88-3c37-4780-adf0-de18fcff10f8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 06 18:43:44.123210 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:44.123192    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/65f72f88-3c37-4780-adf0-de18fcff10f8-istio-token" (OuterVolumeSpecName: "istio-token") pod "65f72f88-3c37-4780-adf0-de18fcff10f8" (UID: "65f72f88-3c37-4780-adf0-de18fcff10f8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 18:43:44.123478 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:44.123461    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/65f72f88-3c37-4780-adf0-de18fcff10f8-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "65f72f88-3c37-4780-adf0-de18fcff10f8" (UID: "65f72f88-3c37-4780-adf0-de18fcff10f8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 18:43:44.123525 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:44.123486    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/65f72f88-3c37-4780-adf0-de18fcff10f8-kube-api-access-n8wvc" (OuterVolumeSpecName: "kube-api-access-n8wvc") pod "65f72f88-3c37-4780-adf0-de18fcff10f8" (UID: "65f72f88-3c37-4780-adf0-de18fcff10f8"). InnerVolumeSpecName "kube-api-access-n8wvc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 18:43:44.222650 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:44.222615    2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/65f72f88-3c37-4780-adf0-de18fcff10f8-istio-podinfo\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:43:44.222650 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:44.222643    2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/65f72f88-3c37-4780-adf0-de18fcff10f8-istiod-ca-cert\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:43:44.222650 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:44.222654    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n8wvc\" (UniqueName: \"kubernetes.io/projected/65f72f88-3c37-4780-adf0-de18fcff10f8-kube-api-access-n8wvc\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:43:44.222862 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:44.222665    2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/65f72f88-3c37-4780-adf0-de18fcff10f8-istio-envoy\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:43:44.222862 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:44.222674    2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/65f72f88-3c37-4780-adf0-de18fcff10f8-workload-socket\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:43:44.222862 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:44.222682    2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/65f72f88-3c37-4780-adf0-de18fcff10f8-credential-socket\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:43:44.222862 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:44.222690    2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/65f72f88-3c37-4780-adf0-de18fcff10f8-istio-token\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:43:44.222862 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:44.222699    2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/65f72f88-3c37-4780-adf0-de18fcff10f8-istio-data\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:43:44.235222 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:44.235187    2580 generic.go:356] "Generic (PLEG): container finished" podID="65f72f88-3c37-4780-adf0-de18fcff10f8" containerID="36682c0702bd8efc2928ed5949c50adc5f37473688c7ced52db08e316c91d08a" exitCode=0
Jul 06 18:43:44.235382 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:44.235236    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-daa9ff09-openshift-default-996c88c-nxzb5"
Jul 06 18:43:44.235382 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:44.235274    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-daa9ff09-openshift-default-996c88c-nxzb5" event={"ID":"65f72f88-3c37-4780-adf0-de18fcff10f8","Type":"ContainerDied","Data":"36682c0702bd8efc2928ed5949c50adc5f37473688c7ced52db08e316c91d08a"}
Jul 06 18:43:44.235382 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:44.235339    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-daa9ff09-openshift-default-996c88c-nxzb5" event={"ID":"65f72f88-3c37-4780-adf0-de18fcff10f8","Type":"ContainerDied","Data":"57c9a0733c8724a355d2060cb20da936670da32c48bb933d6e1623a0a04dd19b"}
Jul 06 18:43:44.235382 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:44.235368    2580 scope.go:117] "RemoveContainer" containerID="36682c0702bd8efc2928ed5949c50adc5f37473688c7ced52db08e316c91d08a"
Jul 06 18:43:44.244264 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:44.244248    2580 scope.go:117] "RemoveContainer" containerID="36682c0702bd8efc2928ed5949c50adc5f37473688c7ced52db08e316c91d08a"
Jul 06 18:43:44.244488 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:43:44.244470    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"36682c0702bd8efc2928ed5949c50adc5f37473688c7ced52db08e316c91d08a\": container with ID starting with 36682c0702bd8efc2928ed5949c50adc5f37473688c7ced52db08e316c91d08a not found: ID does not exist" containerID="36682c0702bd8efc2928ed5949c50adc5f37473688c7ced52db08e316c91d08a"
Jul 06 18:43:44.244548 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:44.244502    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"36682c0702bd8efc2928ed5949c50adc5f37473688c7ced52db08e316c91d08a"} err="failed to get container status \"36682c0702bd8efc2928ed5949c50adc5f37473688c7ced52db08e316c91d08a\": rpc error: code = NotFound desc = could not find container \"36682c0702bd8efc2928ed5949c50adc5f37473688c7ced52db08e316c91d08a\": container with ID starting with 36682c0702bd8efc2928ed5949c50adc5f37473688c7ced52db08e316c91d08a not found: ID does not exist"
Jul 06 18:43:44.265574 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:44.265534    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-daa9ff09-openshift-default-996c88c-nxzb5"]
Jul 06 18:43:44.269171 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:44.269142    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-6a1e6982-openshift-default-fff9976dc-24sgx"]
Jul 06 18:43:44.269539 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:44.269521    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="65f72f88-3c37-4780-adf0-de18fcff10f8" containerName="istio-proxy"
Jul 06 18:43:44.269539 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:44.269538    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="65f72f88-3c37-4780-adf0-de18fcff10f8" containerName="istio-proxy"
Jul 06 18:43:44.269702 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:44.269557    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e9f4b6d-8898-41c6-b490-7328e50223d3" containerName="maas-api"
Jul 06 18:43:44.269702 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:44.269566    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e9f4b6d-8898-41c6-b490-7328e50223d3" containerName="maas-api"
Jul 06 18:43:44.269702 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:44.269654    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="4e9f4b6d-8898-41c6-b490-7328e50223d3" containerName="maas-api"
Jul 06 18:43:44.269702 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:44.269667    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="65f72f88-3c37-4780-adf0-de18fcff10f8" containerName="istio-proxy"
Jul 06 18:43:44.272511 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:44.272489    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-6a1e6982-openshift-default-fff9976dc-24sgx"
Jul 06 18:43:44.275247 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:44.275226    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-6a1e6982-openshift-default-dockercfg-4r4lb\""
Jul 06 18:43:44.279417 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:44.279394    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-daa9ff09-openshift-default-996c88c-nxzb5"]
Jul 06 18:43:44.285608 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:44.285580    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-6a1e6982-openshift-default-fff9976dc-24sgx"]
Jul 06 18:43:44.323763 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:44.323734    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/220ab54e-b559-402a-a840-1d83830a5b7c-istiod-ca-cert\") pod \"e2e-ait-6a1e6982-openshift-default-fff9976dc-24sgx\" (UID: \"220ab54e-b559-402a-a840-1d83830a5b7c\") " pod="openshift-ingress/e2e-ait-6a1e6982-openshift-default-fff9976dc-24sgx"
Jul 06 18:43:44.323921 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:44.323778    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/220ab54e-b559-402a-a840-1d83830a5b7c-istio-data\") pod \"e2e-ait-6a1e6982-openshift-default-fff9976dc-24sgx\" (UID: \"220ab54e-b559-402a-a840-1d83830a5b7c\") " pod="openshift-ingress/e2e-ait-6a1e6982-openshift-default-fff9976dc-24sgx"
Jul 06 18:43:44.323921 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:44.323801    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/220ab54e-b559-402a-a840-1d83830a5b7c-workload-socket\") pod \"e2e-ait-6a1e6982-openshift-default-fff9976dc-24sgx\" (UID: \"220ab54e-b559-402a-a840-1d83830a5b7c\") " pod="openshift-ingress/e2e-ait-6a1e6982-openshift-default-fff9976dc-24sgx"
Jul 06 18:43:44.323921 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:44.323882    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/220ab54e-b559-402a-a840-1d83830a5b7c-istio-token\") pod \"e2e-ait-6a1e6982-openshift-default-fff9976dc-24sgx\" (UID: \"220ab54e-b559-402a-a840-1d83830a5b7c\") " pod="openshift-ingress/e2e-ait-6a1e6982-openshift-default-fff9976dc-24sgx"
Jul 06 18:43:44.324069 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:44.323920    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/220ab54e-b559-402a-a840-1d83830a5b7c-credential-socket\") pod \"e2e-ait-6a1e6982-openshift-default-fff9976dc-24sgx\" (UID: \"220ab54e-b559-402a-a840-1d83830a5b7c\") " pod="openshift-ingress/e2e-ait-6a1e6982-openshift-default-fff9976dc-24sgx"
Jul 06 18:43:44.324069 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:44.323940    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/220ab54e-b559-402a-a840-1d83830a5b7c-workload-certs\") pod \"e2e-ait-6a1e6982-openshift-default-fff9976dc-24sgx\" (UID: \"220ab54e-b559-402a-a840-1d83830a5b7c\") " pod="openshift-ingress/e2e-ait-6a1e6982-openshift-default-fff9976dc-24sgx"
Jul 06 18:43:44.324069 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:44.323955    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/220ab54e-b559-402a-a840-1d83830a5b7c-istio-envoy\") pod \"e2e-ait-6a1e6982-openshift-default-fff9976dc-24sgx\" (UID: \"220ab54e-b559-402a-a840-1d83830a5b7c\") " pod="openshift-ingress/e2e-ait-6a1e6982-openshift-default-fff9976dc-24sgx"
Jul 06 18:43:44.324069 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:44.323977    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f7fb7\" (UniqueName: \"kubernetes.io/projected/220ab54e-b559-402a-a840-1d83830a5b7c-kube-api-access-f7fb7\") pod \"e2e-ait-6a1e6982-openshift-default-fff9976dc-24sgx\" (UID: \"220ab54e-b559-402a-a840-1d83830a5b7c\") " pod="openshift-ingress/e2e-ait-6a1e6982-openshift-default-fff9976dc-24sgx"
Jul 06 18:43:44.324212 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:44.324086    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/220ab54e-b559-402a-a840-1d83830a5b7c-istio-podinfo\") pod \"e2e-ait-6a1e6982-openshift-default-fff9976dc-24sgx\" (UID: \"220ab54e-b559-402a-a840-1d83830a5b7c\") " pod="openshift-ingress/e2e-ait-6a1e6982-openshift-default-fff9976dc-24sgx"
Jul 06 18:43:44.425104 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:44.425067    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f7fb7\" (UniqueName: \"kubernetes.io/projected/220ab54e-b559-402a-a840-1d83830a5b7c-kube-api-access-f7fb7\") pod \"e2e-ait-6a1e6982-openshift-default-fff9976dc-24sgx\" (UID: \"220ab54e-b559-402a-a840-1d83830a5b7c\") " pod="openshift-ingress/e2e-ait-6a1e6982-openshift-default-fff9976dc-24sgx"
Jul 06 18:43:44.425286 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:44.425159    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/220ab54e-b559-402a-a840-1d83830a5b7c-istio-podinfo\") pod \"e2e-ait-6a1e6982-openshift-default-fff9976dc-24sgx\" (UID: \"220ab54e-b559-402a-a840-1d83830a5b7c\") " pod="openshift-ingress/e2e-ait-6a1e6982-openshift-default-fff9976dc-24sgx"
Jul 06 18:43:44.425286 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:44.425190    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/220ab54e-b559-402a-a840-1d83830a5b7c-istiod-ca-cert\") pod \"e2e-ait-6a1e6982-openshift-default-fff9976dc-24sgx\" (UID: \"220ab54e-b559-402a-a840-1d83830a5b7c\") " pod="openshift-ingress/e2e-ait-6a1e6982-openshift-default-fff9976dc-24sgx"
Jul 06 18:43:44.425286 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:44.425240    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/220ab54e-b559-402a-a840-1d83830a5b7c-istio-data\") pod \"e2e-ait-6a1e6982-openshift-default-fff9976dc-24sgx\" (UID: \"220ab54e-b559-402a-a840-1d83830a5b7c\") " pod="openshift-ingress/e2e-ait-6a1e6982-openshift-default-fff9976dc-24sgx"
Jul 06 18:43:44.425286 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:44.425273    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/220ab54e-b559-402a-a840-1d83830a5b7c-workload-socket\") pod \"e2e-ait-6a1e6982-openshift-default-fff9976dc-24sgx\" (UID: \"220ab54e-b559-402a-a840-1d83830a5b7c\") " pod="openshift-ingress/e2e-ait-6a1e6982-openshift-default-fff9976dc-24sgx"
Jul 06 18:43:44.425543 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:44.425310    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/220ab54e-b559-402a-a840-1d83830a5b7c-istio-token\") pod \"e2e-ait-6a1e6982-openshift-default-fff9976dc-24sgx\" (UID: \"220ab54e-b559-402a-a840-1d83830a5b7c\") " pod="openshift-ingress/e2e-ait-6a1e6982-openshift-default-fff9976dc-24sgx"
Jul 06 18:43:44.425543 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:44.425351    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/220ab54e-b559-402a-a840-1d83830a5b7c-credential-socket\") pod \"e2e-ait-6a1e6982-openshift-default-fff9976dc-24sgx\" (UID: \"220ab54e-b559-402a-a840-1d83830a5b7c\") " pod="openshift-ingress/e2e-ait-6a1e6982-openshift-default-fff9976dc-24sgx"
Jul 06 18:43:44.425543 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:44.425377    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/220ab54e-b559-402a-a840-1d83830a5b7c-workload-certs\") pod \"e2e-ait-6a1e6982-openshift-default-fff9976dc-24sgx\" (UID: \"220ab54e-b559-402a-a840-1d83830a5b7c\") " pod="openshift-ingress/e2e-ait-6a1e6982-openshift-default-fff9976dc-24sgx"
Jul 06 18:43:44.425543 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:44.425400    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/220ab54e-b559-402a-a840-1d83830a5b7c-istio-envoy\") pod \"e2e-ait-6a1e6982-openshift-default-fff9976dc-24sgx\" (UID: \"220ab54e-b559-402a-a840-1d83830a5b7c\") " pod="openshift-ingress/e2e-ait-6a1e6982-openshift-default-fff9976dc-24sgx"
Jul 06 18:43:44.425752 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:44.425719    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/220ab54e-b559-402a-a840-1d83830a5b7c-workload-socket\") pod \"e2e-ait-6a1e6982-openshift-default-fff9976dc-24sgx\" (UID: \"220ab54e-b559-402a-a840-1d83830a5b7c\") " pod="openshift-ingress/e2e-ait-6a1e6982-openshift-default-fff9976dc-24sgx"
Jul 06 18:43:44.425809 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:44.425787    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/220ab54e-b559-402a-a840-1d83830a5b7c-istio-data\") pod \"e2e-ait-6a1e6982-openshift-default-fff9976dc-24sgx\" (UID: \"220ab54e-b559-402a-a840-1d83830a5b7c\") " pod="openshift-ingress/e2e-ait-6a1e6982-openshift-default-fff9976dc-24sgx"
Jul 06 18:43:44.425988 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:44.425958    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/220ab54e-b559-402a-a840-1d83830a5b7c-credential-socket\") pod \"e2e-ait-6a1e6982-openshift-default-fff9976dc-24sgx\" (UID: \"220ab54e-b559-402a-a840-1d83830a5b7c\") " pod="openshift-ingress/e2e-ait-6a1e6982-openshift-default-fff9976dc-24sgx"
Jul 06 18:43:44.426156 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:44.426010    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/220ab54e-b559-402a-a840-1d83830a5b7c-workload-certs\") pod \"e2e-ait-6a1e6982-openshift-default-fff9976dc-24sgx\" (UID: \"220ab54e-b559-402a-a840-1d83830a5b7c\") " pod="openshift-ingress/e2e-ait-6a1e6982-openshift-default-fff9976dc-24sgx"
Jul 06 18:43:44.426156 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:44.426091    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/220ab54e-b559-402a-a840-1d83830a5b7c-istiod-ca-cert\") pod \"e2e-ait-6a1e6982-openshift-default-fff9976dc-24sgx\" (UID: \"220ab54e-b559-402a-a840-1d83830a5b7c\") " pod="openshift-ingress/e2e-ait-6a1e6982-openshift-default-fff9976dc-24sgx"
Jul 06 18:43:44.427547 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:44.427530    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/220ab54e-b559-402a-a840-1d83830a5b7c-istio-envoy\") pod \"e2e-ait-6a1e6982-openshift-default-fff9976dc-24sgx\" (UID: \"220ab54e-b559-402a-a840-1d83830a5b7c\") " pod="openshift-ingress/e2e-ait-6a1e6982-openshift-default-fff9976dc-24sgx"
Jul 06 18:43:44.427942 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:44.427927    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/220ab54e-b559-402a-a840-1d83830a5b7c-istio-podinfo\") pod \"e2e-ait-6a1e6982-openshift-default-fff9976dc-24sgx\" (UID: \"220ab54e-b559-402a-a840-1d83830a5b7c\") " pod="openshift-ingress/e2e-ait-6a1e6982-openshift-default-fff9976dc-24sgx"
Jul 06 18:43:44.435199 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:44.435172    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/220ab54e-b559-402a-a840-1d83830a5b7c-istio-token\") pod \"e2e-ait-6a1e6982-openshift-default-fff9976dc-24sgx\" (UID: \"220ab54e-b559-402a-a840-1d83830a5b7c\") " pod="openshift-ingress/e2e-ait-6a1e6982-openshift-default-fff9976dc-24sgx"
Jul 06 18:43:44.435306 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:44.435291    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f7fb7\" (UniqueName: \"kubernetes.io/projected/220ab54e-b559-402a-a840-1d83830a5b7c-kube-api-access-f7fb7\") pod \"e2e-ait-6a1e6982-openshift-default-fff9976dc-24sgx\" (UID: \"220ab54e-b559-402a-a840-1d83830a5b7c\") " pod="openshift-ingress/e2e-ait-6a1e6982-openshift-default-fff9976dc-24sgx"
Jul 06 18:43:44.584345 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:44.584321    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-6a1e6982-openshift-default-fff9976dc-24sgx"
Jul 06 18:43:44.723466 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:44.723439    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-6a1e6982-openshift-default-fff9976dc-24sgx"]
Jul 06 18:43:44.725123 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:43:44.725092    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod220ab54e_b559_402a_a840_1d83830a5b7c.slice/crio-ea8320c1d03e179fbd6eaa9b6a288cab5bbdbfc54f271aa3ce11f47c49adbfad WatchSource:0}: Error finding container ea8320c1d03e179fbd6eaa9b6a288cab5bbdbfc54f271aa3ce11f47c49adbfad: Status 404 returned error can't find the container with id ea8320c1d03e179fbd6eaa9b6a288cab5bbdbfc54f271aa3ce11f47c49adbfad
Jul 06 18:43:44.727226 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:44.727192    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 06 18:43:44.727314 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:44.727254    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 06 18:43:44.727314 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:44.727286    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 06 18:43:45.241226 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:45.241179    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-6a1e6982-openshift-default-fff9976dc-24sgx" event={"ID":"220ab54e-b559-402a-a840-1d83830a5b7c","Type":"ContainerStarted","Data":"c354e11bb6525a7ea7157528f14c41342b56fb4039cba0ddf09c77872a8dba9a"}
Jul 06 18:43:45.241629 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:45.241255    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-6a1e6982-openshift-default-fff9976dc-24sgx" event={"ID":"220ab54e-b559-402a-a840-1d83830a5b7c","Type":"ContainerStarted","Data":"ea8320c1d03e179fbd6eaa9b6a288cab5bbdbfc54f271aa3ce11f47c49adbfad"}
Jul 06 18:43:45.261915 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:45.261860    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-6a1e6982-openshift-default-fff9976dc-24sgx" podStartSLOduration=1.261843805 podStartE2EDuration="1.261843805s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:43:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:43:45.261424208 +0000 UTC m=+1560.418822694" watchObservedRunningTime="2026-07-06 18:43:45.261843805 +0000 UTC m=+1560.419242290"
Jul 06 18:43:45.323719 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:45.323679    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="65f72f88-3c37-4780-adf0-de18fcff10f8" path="/var/lib/kubelet/pods/65f72f88-3c37-4780-adf0-de18fcff10f8/volumes"
Jul 06 18:43:45.577388 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:45.577350    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-ait-6a1e6982-7b8775cf8c-fhq9c"]
Jul 06 18:43:45.580993 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:45.580898    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-6a1e6982-7b8775cf8c-fhq9c"
Jul 06 18:43:45.583665 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:45.583640    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-ait-6a1e6982-serving-cert\""
Jul 06 18:43:45.584650 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:45.584630    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-6a1e6982-openshift-default-fff9976dc-24sgx"
Jul 06 18:43:45.591335 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:45.591314    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-6a1e6982-openshift-default-fff9976dc-24sgx"
Jul 06 18:43:45.592226 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:45.592206    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-6a1e6982-7b8775cf8c-fhq9c"]
Jul 06 18:43:45.638248 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:45.638214    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/019b8e80-a1fe-4987-804c-413632557e10-openshift-service-ca\") pod \"maas-api-e2e-ait-6a1e6982-7b8775cf8c-fhq9c\" (UID: \"019b8e80-a1fe-4987-804c-413632557e10\") " pod="opendatahub/maas-api-e2e-ait-6a1e6982-7b8775cf8c-fhq9c"
Jul 06 18:43:45.638456 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:45.638268    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/019b8e80-a1fe-4987-804c-413632557e10-maas-api-tls\") pod \"maas-api-e2e-ait-6a1e6982-7b8775cf8c-fhq9c\" (UID: \"019b8e80-a1fe-4987-804c-413632557e10\") " pod="opendatahub/maas-api-e2e-ait-6a1e6982-7b8775cf8c-fhq9c"
Jul 06 18:43:45.638456 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:45.638350    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mkwmv\" (UniqueName: \"kubernetes.io/projected/019b8e80-a1fe-4987-804c-413632557e10-kube-api-access-mkwmv\") pod \"maas-api-e2e-ait-6a1e6982-7b8775cf8c-fhq9c\" (UID: \"019b8e80-a1fe-4987-804c-413632557e10\") " pod="opendatahub/maas-api-e2e-ait-6a1e6982-7b8775cf8c-fhq9c"
Jul 06 18:43:45.739292 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:45.739247    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/019b8e80-a1fe-4987-804c-413632557e10-openshift-service-ca\") pod \"maas-api-e2e-ait-6a1e6982-7b8775cf8c-fhq9c\" (UID: \"019b8e80-a1fe-4987-804c-413632557e10\") " pod="opendatahub/maas-api-e2e-ait-6a1e6982-7b8775cf8c-fhq9c"
Jul 06 18:43:45.739478 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:45.739307    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/019b8e80-a1fe-4987-804c-413632557e10-maas-api-tls\") pod \"maas-api-e2e-ait-6a1e6982-7b8775cf8c-fhq9c\" (UID: \"019b8e80-a1fe-4987-804c-413632557e10\") " pod="opendatahub/maas-api-e2e-ait-6a1e6982-7b8775cf8c-fhq9c"
Jul 06 18:43:45.739478 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:45.739374    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mkwmv\" (UniqueName: \"kubernetes.io/projected/019b8e80-a1fe-4987-804c-413632557e10-kube-api-access-mkwmv\") pod \"maas-api-e2e-ait-6a1e6982-7b8775cf8c-fhq9c\" (UID: \"019b8e80-a1fe-4987-804c-413632557e10\") " pod="opendatahub/maas-api-e2e-ait-6a1e6982-7b8775cf8c-fhq9c"
Jul 06 18:43:45.739565 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:43:45.739474    2580 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-ait-6a1e6982-serving-cert: secret "maas-api-e2e-ait-6a1e6982-serving-cert" not found
Jul 06 18:43:45.739565 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:43:45.739548    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/019b8e80-a1fe-4987-804c-413632557e10-maas-api-tls podName:019b8e80-a1fe-4987-804c-413632557e10 nodeName:}" failed. No retries permitted until 2026-07-06 18:43:46.239531318 +0000 UTC m=+1561.396929783 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/019b8e80-a1fe-4987-804c-413632557e10-maas-api-tls") pod "maas-api-e2e-ait-6a1e6982-7b8775cf8c-fhq9c" (UID: "019b8e80-a1fe-4987-804c-413632557e10") : secret "maas-api-e2e-ait-6a1e6982-serving-cert" not found
Jul 06 18:43:45.739948 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:45.739928    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/019b8e80-a1fe-4987-804c-413632557e10-openshift-service-ca\") pod \"maas-api-e2e-ait-6a1e6982-7b8775cf8c-fhq9c\" (UID: \"019b8e80-a1fe-4987-804c-413632557e10\") " pod="opendatahub/maas-api-e2e-ait-6a1e6982-7b8775cf8c-fhq9c"
Jul 06 18:43:45.754264 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:45.754235    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mkwmv\" (UniqueName: \"kubernetes.io/projected/019b8e80-a1fe-4987-804c-413632557e10-kube-api-access-mkwmv\") pod \"maas-api-e2e-ait-6a1e6982-7b8775cf8c-fhq9c\" (UID: \"019b8e80-a1fe-4987-804c-413632557e10\") " pod="opendatahub/maas-api-e2e-ait-6a1e6982-7b8775cf8c-fhq9c"
Jul 06 18:43:46.243501 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:46.243467    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/019b8e80-a1fe-4987-804c-413632557e10-maas-api-tls\") pod \"maas-api-e2e-ait-6a1e6982-7b8775cf8c-fhq9c\" (UID: \"019b8e80-a1fe-4987-804c-413632557e10\") " pod="opendatahub/maas-api-e2e-ait-6a1e6982-7b8775cf8c-fhq9c"
Jul 06 18:43:46.246197 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:46.246164    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/019b8e80-a1fe-4987-804c-413632557e10-maas-api-tls\") pod \"maas-api-e2e-ait-6a1e6982-7b8775cf8c-fhq9c\" (UID: \"019b8e80-a1fe-4987-804c-413632557e10\") " pod="opendatahub/maas-api-e2e-ait-6a1e6982-7b8775cf8c-fhq9c"
Jul 06 18:43:46.247460 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:46.247431    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-6a1e6982-openshift-default-fff9976dc-24sgx"
Jul 06 18:43:46.248483 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:46.248465    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-6a1e6982-openshift-default-fff9976dc-24sgx"
Jul 06 18:43:46.496226 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:46.496137    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-6a1e6982-7b8775cf8c-fhq9c"
Jul 06 18:43:46.635575 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:46.635545    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-6a1e6982-7b8775cf8c-fhq9c"]
Jul 06 18:43:46.636987 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:43:46.636965    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod019b8e80_a1fe_4987_804c_413632557e10.slice/crio-d65ee377baf4329c11e346c4f44f2dccb85b437e4d32f884133fda5571a83f5a WatchSource:0}: Error finding container d65ee377baf4329c11e346c4f44f2dccb85b437e4d32f884133fda5571a83f5a: Status 404 returned error can't find the container with id d65ee377baf4329c11e346c4f44f2dccb85b437e4d32f884133fda5571a83f5a
Jul 06 18:43:47.253370 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:47.253270    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-6a1e6982-7b8775cf8c-fhq9c" event={"ID":"019b8e80-a1fe-4987-804c-413632557e10","Type":"ContainerStarted","Data":"d836994abf994a3b0d282c2addf4734a6488a55750be05c8b2467169b35c8acf"}
Jul 06 18:43:47.253370 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:47.253323    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-6a1e6982-7b8775cf8c-fhq9c" event={"ID":"019b8e80-a1fe-4987-804c-413632557e10","Type":"ContainerStarted","Data":"d65ee377baf4329c11e346c4f44f2dccb85b437e4d32f884133fda5571a83f5a"}
Jul 06 18:43:47.253370 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:47.253349    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-ait-6a1e6982-7b8775cf8c-fhq9c"
Jul 06 18:43:47.270980 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:47.270927    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-ait-6a1e6982-7b8775cf8c-fhq9c" podStartSLOduration=1.943865505 podStartE2EDuration="2.270911789s" totalImagesPullingTime="327.046284ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:43:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:43:47.26957389 +0000 UTC m=+1562.426972377" watchObservedRunningTime="2026-07-06 18:43:47.270911789 +0000 UTC m=+1562.428310274"
Jul 06 18:43:50.907476 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:50.907437    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-6a1e6982-7b8775cf8c-fhq9c"]
Jul 06 18:43:50.907964 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:50.907643    2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-ait-6a1e6982-7b8775cf8c-fhq9c" podUID="019b8e80-a1fe-4987-804c-413632557e10" containerName="maas-api" containerID="cri-o://d836994abf994a3b0d282c2addf4734a6488a55750be05c8b2467169b35c8acf" gracePeriod=30
Jul 06 18:43:51.199813 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:51.199789    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-6a1e6982-7b8775cf8c-fhq9c"
Jul 06 18:43:51.272664 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:51.272626    2580 generic.go:356] "Generic (PLEG): container finished" podID="019b8e80-a1fe-4987-804c-413632557e10" containerID="d836994abf994a3b0d282c2addf4734a6488a55750be05c8b2467169b35c8acf" exitCode=0
Jul 06 18:43:51.272830 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:51.272688    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-6a1e6982-7b8775cf8c-fhq9c"
Jul 06 18:43:51.272830 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:51.272717    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-6a1e6982-7b8775cf8c-fhq9c" event={"ID":"019b8e80-a1fe-4987-804c-413632557e10","Type":"ContainerDied","Data":"d836994abf994a3b0d282c2addf4734a6488a55750be05c8b2467169b35c8acf"}
Jul 06 18:43:51.272830 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:51.272816    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-6a1e6982-7b8775cf8c-fhq9c" event={"ID":"019b8e80-a1fe-4987-804c-413632557e10","Type":"ContainerDied","Data":"d65ee377baf4329c11e346c4f44f2dccb85b437e4d32f884133fda5571a83f5a"}
Jul 06 18:43:51.272928 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:51.272852    2580 scope.go:117] "RemoveContainer" containerID="d836994abf994a3b0d282c2addf4734a6488a55750be05c8b2467169b35c8acf"
Jul 06 18:43:51.281167 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:51.281147    2580 scope.go:117] "RemoveContainer" containerID="d836994abf994a3b0d282c2addf4734a6488a55750be05c8b2467169b35c8acf"
Jul 06 18:43:51.281439 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:43:51.281420    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d836994abf994a3b0d282c2addf4734a6488a55750be05c8b2467169b35c8acf\": container with ID starting with d836994abf994a3b0d282c2addf4734a6488a55750be05c8b2467169b35c8acf not found: ID does not exist" containerID="d836994abf994a3b0d282c2addf4734a6488a55750be05c8b2467169b35c8acf"
Jul 06 18:43:51.281489 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:51.281446    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d836994abf994a3b0d282c2addf4734a6488a55750be05c8b2467169b35c8acf"} err="failed to get container status \"d836994abf994a3b0d282c2addf4734a6488a55750be05c8b2467169b35c8acf\": rpc error: code = NotFound desc = could not find container \"d836994abf994a3b0d282c2addf4734a6488a55750be05c8b2467169b35c8acf\": container with ID starting with d836994abf994a3b0d282c2addf4734a6488a55750be05c8b2467169b35c8acf not found: ID does not exist"
Jul 06 18:43:51.288753 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:51.288731    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mkwmv\" (UniqueName: \"kubernetes.io/projected/019b8e80-a1fe-4987-804c-413632557e10-kube-api-access-mkwmv\") pod \"019b8e80-a1fe-4987-804c-413632557e10\" (UID: \"019b8e80-a1fe-4987-804c-413632557e10\") "
Jul 06 18:43:51.288947 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:51.288810    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/019b8e80-a1fe-4987-804c-413632557e10-maas-api-tls\") pod \"019b8e80-a1fe-4987-804c-413632557e10\" (UID: \"019b8e80-a1fe-4987-804c-413632557e10\") "
Jul 06 18:43:51.288947 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:51.288866    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/019b8e80-a1fe-4987-804c-413632557e10-openshift-service-ca\") pod \"019b8e80-a1fe-4987-804c-413632557e10\" (UID: \"019b8e80-a1fe-4987-804c-413632557e10\") "
Jul 06 18:43:51.289246 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:51.289224    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/019b8e80-a1fe-4987-804c-413632557e10-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "019b8e80-a1fe-4987-804c-413632557e10" (UID: "019b8e80-a1fe-4987-804c-413632557e10"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 18:43:51.290669 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:51.290650    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/019b8e80-a1fe-4987-804c-413632557e10-kube-api-access-mkwmv" (OuterVolumeSpecName: "kube-api-access-mkwmv") pod "019b8e80-a1fe-4987-804c-413632557e10" (UID: "019b8e80-a1fe-4987-804c-413632557e10"). InnerVolumeSpecName "kube-api-access-mkwmv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 18:43:51.290747 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:51.290730    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/019b8e80-a1fe-4987-804c-413632557e10-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "019b8e80-a1fe-4987-804c-413632557e10" (UID: "019b8e80-a1fe-4987-804c-413632557e10"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 18:43:51.389937 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:51.389899    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/019b8e80-a1fe-4987-804c-413632557e10-openshift-service-ca\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:43:51.389937 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:51.389932    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mkwmv\" (UniqueName: \"kubernetes.io/projected/019b8e80-a1fe-4987-804c-413632557e10-kube-api-access-mkwmv\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:43:51.389937 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:51.389942    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/019b8e80-a1fe-4987-804c-413632557e10-maas-api-tls\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:43:51.592583 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:51.592552    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-6a1e6982-7b8775cf8c-fhq9c"]
Jul 06 18:43:51.598739 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:51.598713    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-ait-6a1e6982-7b8775cf8c-fhq9c"]
Jul 06 18:43:53.323602 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:53.323574    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="019b8e80-a1fe-4987-804c-413632557e10" path="/var/lib/kubelet/pods/019b8e80-a1fe-4987-804c-413632557e10/volumes"
Jul 06 18:43:56.701369 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:56.701325    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-6a1e6982-openshift-default-fff9976dc-24sgx"]
Jul 06 18:43:56.701848 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:56.701576    2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-6a1e6982-openshift-default-fff9976dc-24sgx" podUID="220ab54e-b559-402a-a840-1d83830a5b7c" containerName="istio-proxy" containerID="cri-o://c354e11bb6525a7ea7157528f14c41342b56fb4039cba0ddf09c77872a8dba9a" gracePeriod=30
Jul 06 18:43:57.059660 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:57.059635    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-6a1e6982-openshift-default-fff9976dc-24sgx"
Jul 06 18:43:57.142274 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:57.142243    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/220ab54e-b559-402a-a840-1d83830a5b7c-istio-podinfo\") pod \"220ab54e-b559-402a-a840-1d83830a5b7c\" (UID: \"220ab54e-b559-402a-a840-1d83830a5b7c\") "
Jul 06 18:43:57.142432 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:57.142331    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f7fb7\" (UniqueName: \"kubernetes.io/projected/220ab54e-b559-402a-a840-1d83830a5b7c-kube-api-access-f7fb7\") pod \"220ab54e-b559-402a-a840-1d83830a5b7c\" (UID: \"220ab54e-b559-402a-a840-1d83830a5b7c\") "
Jul 06 18:43:57.142432 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:57.142396    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/220ab54e-b559-402a-a840-1d83830a5b7c-credential-socket\") pod \"220ab54e-b559-402a-a840-1d83830a5b7c\" (UID: \"220ab54e-b559-402a-a840-1d83830a5b7c\") "
Jul 06 18:43:57.142535 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:57.142515    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/220ab54e-b559-402a-a840-1d83830a5b7c-istiod-ca-cert\") pod \"220ab54e-b559-402a-a840-1d83830a5b7c\" (UID: \"220ab54e-b559-402a-a840-1d83830a5b7c\") "
Jul 06 18:43:57.142610 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:57.142587    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/220ab54e-b559-402a-a840-1d83830a5b7c-istio-envoy\") pod \"220ab54e-b559-402a-a840-1d83830a5b7c\" (UID: \"220ab54e-b559-402a-a840-1d83830a5b7c\") "
Jul 06 18:43:57.142674 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:57.142636    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/220ab54e-b559-402a-a840-1d83830a5b7c-istio-data\") pod \"220ab54e-b559-402a-a840-1d83830a5b7c\" (UID: \"220ab54e-b559-402a-a840-1d83830a5b7c\") "
Jul 06 18:43:57.142674 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:57.142645    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/220ab54e-b559-402a-a840-1d83830a5b7c-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "220ab54e-b559-402a-a840-1d83830a5b7c" (UID: "220ab54e-b559-402a-a840-1d83830a5b7c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 18:43:57.142674 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:57.142661    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/220ab54e-b559-402a-a840-1d83830a5b7c-workload-certs\") pod \"220ab54e-b559-402a-a840-1d83830a5b7c\" (UID: \"220ab54e-b559-402a-a840-1d83830a5b7c\") "
Jul 06 18:43:57.142820 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:57.142691    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/220ab54e-b559-402a-a840-1d83830a5b7c-istio-token\") pod \"220ab54e-b559-402a-a840-1d83830a5b7c\" (UID: \"220ab54e-b559-402a-a840-1d83830a5b7c\") "
Jul 06 18:43:57.142820 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:57.142733    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/220ab54e-b559-402a-a840-1d83830a5b7c-workload-socket\") pod \"220ab54e-b559-402a-a840-1d83830a5b7c\" (UID: \"220ab54e-b559-402a-a840-1d83830a5b7c\") "
Jul 06 18:43:57.142916 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:57.142884    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/220ab54e-b559-402a-a840-1d83830a5b7c-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "220ab54e-b559-402a-a840-1d83830a5b7c" (UID: "220ab54e-b559-402a-a840-1d83830a5b7c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 18:43:57.143204 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:57.143124    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/220ab54e-b559-402a-a840-1d83830a5b7c-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "220ab54e-b559-402a-a840-1d83830a5b7c" (UID: "220ab54e-b559-402a-a840-1d83830a5b7c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 18:43:57.143204 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:57.143134    2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/220ab54e-b559-402a-a840-1d83830a5b7c-credential-socket\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:43:57.143204 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:57.143157    2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/220ab54e-b559-402a-a840-1d83830a5b7c-istiod-ca-cert\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:43:57.143368 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:57.143212    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/220ab54e-b559-402a-a840-1d83830a5b7c-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "220ab54e-b559-402a-a840-1d83830a5b7c" (UID: "220ab54e-b559-402a-a840-1d83830a5b7c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 18:43:57.143757 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:57.143734    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/220ab54e-b559-402a-a840-1d83830a5b7c-istio-data" (OuterVolumeSpecName: "istio-data") pod "220ab54e-b559-402a-a840-1d83830a5b7c" (UID: "220ab54e-b559-402a-a840-1d83830a5b7c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 18:43:57.144837 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:57.144815    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/220ab54e-b559-402a-a840-1d83830a5b7c-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "220ab54e-b559-402a-a840-1d83830a5b7c" (UID: "220ab54e-b559-402a-a840-1d83830a5b7c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 06 18:43:57.144923 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:57.144848    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/220ab54e-b559-402a-a840-1d83830a5b7c-kube-api-access-f7fb7" (OuterVolumeSpecName: "kube-api-access-f7fb7") pod "220ab54e-b559-402a-a840-1d83830a5b7c" (UID: "220ab54e-b559-402a-a840-1d83830a5b7c"). InnerVolumeSpecName "kube-api-access-f7fb7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 18:43:57.144923 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:57.144852    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/220ab54e-b559-402a-a840-1d83830a5b7c-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "220ab54e-b559-402a-a840-1d83830a5b7c" (UID: "220ab54e-b559-402a-a840-1d83830a5b7c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 18:43:57.145170 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:57.145149    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/220ab54e-b559-402a-a840-1d83830a5b7c-istio-token" (OuterVolumeSpecName: "istio-token") pod "220ab54e-b559-402a-a840-1d83830a5b7c" (UID: "220ab54e-b559-402a-a840-1d83830a5b7c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 18:43:57.244168 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:57.244055    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f7fb7\" (UniqueName: \"kubernetes.io/projected/220ab54e-b559-402a-a840-1d83830a5b7c-kube-api-access-f7fb7\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:43:57.244168 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:57.244103    2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/220ab54e-b559-402a-a840-1d83830a5b7c-istio-envoy\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:43:57.244168 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:57.244116    2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/220ab54e-b559-402a-a840-1d83830a5b7c-istio-data\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:43:57.244168 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:57.244129    2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/220ab54e-b559-402a-a840-1d83830a5b7c-workload-certs\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:43:57.244168 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:57.244139    2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/220ab54e-b559-402a-a840-1d83830a5b7c-istio-token\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:43:57.244168 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:57.244151    2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/220ab54e-b559-402a-a840-1d83830a5b7c-workload-socket\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:43:57.244489 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:57.244194    2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/220ab54e-b559-402a-a840-1d83830a5b7c-istio-podinfo\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:43:57.305192 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:57.305155    2580 generic.go:356] "Generic (PLEG): container finished" podID="220ab54e-b559-402a-a840-1d83830a5b7c" containerID="c354e11bb6525a7ea7157528f14c41342b56fb4039cba0ddf09c77872a8dba9a" exitCode=0
Jul 06 18:43:57.305445 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:57.305215    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-6a1e6982-openshift-default-fff9976dc-24sgx"
Jul 06 18:43:57.305445 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:57.305240    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-6a1e6982-openshift-default-fff9976dc-24sgx" event={"ID":"220ab54e-b559-402a-a840-1d83830a5b7c","Type":"ContainerDied","Data":"c354e11bb6525a7ea7157528f14c41342b56fb4039cba0ddf09c77872a8dba9a"}
Jul 06 18:43:57.305445 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:57.305299    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-6a1e6982-openshift-default-fff9976dc-24sgx" event={"ID":"220ab54e-b559-402a-a840-1d83830a5b7c","Type":"ContainerDied","Data":"ea8320c1d03e179fbd6eaa9b6a288cab5bbdbfc54f271aa3ce11f47c49adbfad"}
Jul 06 18:43:57.305445 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:57.305324    2580 scope.go:117] "RemoveContainer" containerID="c354e11bb6525a7ea7157528f14c41342b56fb4039cba0ddf09c77872a8dba9a"
Jul 06 18:43:57.316301 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:57.316106    2580 scope.go:117] "RemoveContainer" containerID="c354e11bb6525a7ea7157528f14c41342b56fb4039cba0ddf09c77872a8dba9a"
Jul 06 18:43:57.316483 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:43:57.316453    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c354e11bb6525a7ea7157528f14c41342b56fb4039cba0ddf09c77872a8dba9a\": container with ID starting with c354e11bb6525a7ea7157528f14c41342b56fb4039cba0ddf09c77872a8dba9a not found: ID does not exist" containerID="c354e11bb6525a7ea7157528f14c41342b56fb4039cba0ddf09c77872a8dba9a"
Jul 06 18:43:57.316601 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:57.316495    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c354e11bb6525a7ea7157528f14c41342b56fb4039cba0ddf09c77872a8dba9a"} err="failed to get container status \"c354e11bb6525a7ea7157528f14c41342b56fb4039cba0ddf09c77872a8dba9a\": rpc error: code = NotFound desc = could not find container \"c354e11bb6525a7ea7157528f14c41342b56fb4039cba0ddf09c77872a8dba9a\": container with ID starting with c354e11bb6525a7ea7157528f14c41342b56fb4039cba0ddf09c77872a8dba9a not found: ID does not exist"
Jul 06 18:43:57.330337 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:57.330308    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-6a1e6982-openshift-default-fff9976dc-24sgx"]
Jul 06 18:43:57.336228 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:57.336207    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-6a1e6982-openshift-default-fff9976dc-24sgx"]
Jul 06 18:43:59.322379 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:43:59.322340    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="220ab54e-b559-402a-a840-1d83830a5b7c" path="/var/lib/kubelet/pods/220ab54e-b559-402a-a840-1d83830a5b7c/volumes"
Jul 06 18:44:02.933133 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:44:02.933097    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-9f07f5b1-openshift-default-cd6f4f995-lxxs7"]
Jul 06 18:44:02.933658 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:44:02.933518    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="019b8e80-a1fe-4987-804c-413632557e10" containerName="maas-api"
Jul 06 18:44:02.933658 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:44:02.933532    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="019b8e80-a1fe-4987-804c-413632557e10" containerName="maas-api"
Jul 06 18:44:02.933658 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:44:02.933544    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="220ab54e-b559-402a-a840-1d83830a5b7c" containerName="istio-proxy"
Jul 06 18:44:02.933658 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:44:02.933549    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="220ab54e-b559-402a-a840-1d83830a5b7c" containerName="istio-proxy"
Jul 06 18:44:02.933658 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:44:02.933625    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="220ab54e-b559-402a-a840-1d83830a5b7c" containerName="istio-proxy"
Jul 06 18:44:02.933658 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:44:02.933636    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="019b8e80-a1fe-4987-804c-413632557e10" containerName="maas-api"
Jul 06 18:44:02.936554 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:44:02.936533    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-9f07f5b1-openshift-default-cd6f4f995-lxxs7"
Jul 06 18:44:02.941277 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:44:02.941236    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-9f07f5b1-openshift-default-dockercfg-pg6jg\""
Jul 06 18:44:02.949826 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:44:02.949797    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-9f07f5b1-openshift-default-cd6f4f995-lxxs7"]
Jul 06 18:44:02.997754 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:44:02.997718    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/755e8837-714b-455b-b3f6-0d03464aa561-workload-certs\") pod \"e2e-derive-9f07f5b1-openshift-default-cd6f4f995-lxxs7\" (UID: \"755e8837-714b-455b-b3f6-0d03464aa561\") " pod="openshift-ingress/e2e-derive-9f07f5b1-openshift-default-cd6f4f995-lxxs7"
Jul 06 18:44:02.997948 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:44:02.997769    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/755e8837-714b-455b-b3f6-0d03464aa561-istio-token\") pod \"e2e-derive-9f07f5b1-openshift-default-cd6f4f995-lxxs7\" (UID: \"755e8837-714b-455b-b3f6-0d03464aa561\") " pod="openshift-ingress/e2e-derive-9f07f5b1-openshift-default-cd6f4f995-lxxs7"
Jul 06 18:44:02.997948 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:44:02.997791    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/755e8837-714b-455b-b3f6-0d03464aa561-istio-podinfo\") pod \"e2e-derive-9f07f5b1-openshift-default-cd6f4f995-lxxs7\" (UID: \"755e8837-714b-455b-b3f6-0d03464aa561\") " pod="openshift-ingress/e2e-derive-9f07f5b1-openshift-default-cd6f4f995-lxxs7"
Jul 06 18:44:02.997948 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:44:02.997835    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/755e8837-714b-455b-b3f6-0d03464aa561-istio-data\") pod \"e2e-derive-9f07f5b1-openshift-default-cd6f4f995-lxxs7\" (UID: \"755e8837-714b-455b-b3f6-0d03464aa561\") " pod="openshift-ingress/e2e-derive-9f07f5b1-openshift-default-cd6f4f995-lxxs7"
Jul 06 18:44:02.997948 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:44:02.997874    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/755e8837-714b-455b-b3f6-0d03464aa561-credential-socket\") pod \"e2e-derive-9f07f5b1-openshift-default-cd6f4f995-lxxs7\" (UID: \"755e8837-714b-455b-b3f6-0d03464aa561\") " pod="openshift-ingress/e2e-derive-9f07f5b1-openshift-default-cd6f4f995-lxxs7"
Jul 06 18:44:02.997948 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:44:02.997901    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/755e8837-714b-455b-b3f6-0d03464aa561-istio-envoy\") pod \"e2e-derive-9f07f5b1-openshift-default-cd6f4f995-lxxs7\" (UID: \"755e8837-714b-455b-b3f6-0d03464aa561\") " pod="openshift-ingress/e2e-derive-9f07f5b1-openshift-default-cd6f4f995-lxxs7"
Jul 06 18:44:02.997948 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:44:02.997935    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/755e8837-714b-455b-b3f6-0d03464aa561-workload-socket\") pod \"e2e-derive-9f07f5b1-openshift-default-cd6f4f995-lxxs7\" (UID: \"755e8837-714b-455b-b3f6-0d03464aa561\") " pod="openshift-ingress/e2e-derive-9f07f5b1-openshift-default-cd6f4f995-lxxs7"
Jul 06 18:44:02.998195 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:44:02.997953    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/755e8837-714b-455b-b3f6-0d03464aa561-istiod-ca-cert\") pod \"e2e-derive-9f07f5b1-openshift-default-cd6f4f995-lxxs7\" (UID: \"755e8837-714b-455b-b3f6-0d03464aa561\") " pod="openshift-ingress/e2e-derive-9f07f5b1-openshift-default-cd6f4f995-lxxs7"
Jul 06 18:44:02.998195 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:44:02.998000    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5gp6w\" (UniqueName: \"kubernetes.io/projected/755e8837-714b-455b-b3f6-0d03464aa561-kube-api-access-5gp6w\") pod \"e2e-derive-9f07f5b1-openshift-default-cd6f4f995-lxxs7\" (UID: \"755e8837-714b-455b-b3f6-0d03464aa561\") " pod="openshift-ingress/e2e-derive-9f07f5b1-openshift-default-cd6f4f995-lxxs7"
Jul 06 18:44:03.099284 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:44:03.099234    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/755e8837-714b-455b-b3f6-0d03464aa561-workload-socket\") pod \"e2e-derive-9f07f5b1-openshift-default-cd6f4f995-lxxs7\" (UID: \"755e8837-714b-455b-b3f6-0d03464aa561\") " pod="openshift-ingress/e2e-derive-9f07f5b1-openshift-default-cd6f4f995-lxxs7"
Jul 06 18:44:03.099284 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:44:03.099270    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/755e8837-714b-455b-b3f6-0d03464aa561-istiod-ca-cert\") pod \"e2e-derive-9f07f5b1-openshift-default-cd6f4f995-lxxs7\" (UID: \"755e8837-714b-455b-b3f6-0d03464aa561\") " pod="openshift-ingress/e2e-derive-9f07f5b1-openshift-default-cd6f4f995-lxxs7"
Jul 06 18:44:03.099549 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:44:03.099310    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5gp6w\" (UniqueName: \"kubernetes.io/projected/755e8837-714b-455b-b3f6-0d03464aa561-kube-api-access-5gp6w\") pod \"e2e-derive-9f07f5b1-openshift-default-cd6f4f995-lxxs7\" (UID: \"755e8837-714b-455b-b3f6-0d03464aa561\") " pod="openshift-ingress/e2e-derive-9f07f5b1-openshift-default-cd6f4f995-lxxs7"
Jul 06 18:44:03.099549 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:44:03.099355    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/755e8837-714b-455b-b3f6-0d03464aa561-workload-certs\") pod \"e2e-derive-9f07f5b1-openshift-default-cd6f4f995-lxxs7\" (UID: \"755e8837-714b-455b-b3f6-0d03464aa561\") " pod="openshift-ingress/e2e-derive-9f07f5b1-openshift-default-cd6f4f995-lxxs7"
Jul 06 18:44:03.099549 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:44:03.099390    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/755e8837-714b-455b-b3f6-0d03464aa561-istio-token\") pod \"e2e-derive-9f07f5b1-openshift-default-cd6f4f995-lxxs7\" (UID: \"755e8837-714b-455b-b3f6-0d03464aa561\") " pod="openshift-ingress/e2e-derive-9f07f5b1-openshift-default-cd6f4f995-lxxs7"
Jul 06 18:44:03.099549 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:44:03.099411    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/755e8837-714b-455b-b3f6-0d03464aa561-istio-podinfo\") pod \"e2e-derive-9f07f5b1-openshift-default-cd6f4f995-lxxs7\" (UID: \"755e8837-714b-455b-b3f6-0d03464aa561\") " pod="openshift-ingress/e2e-derive-9f07f5b1-openshift-default-cd6f4f995-lxxs7"
Jul 06 18:44:03.099549 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:44:03.099511    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/755e8837-714b-455b-b3f6-0d03464aa561-istio-data\") pod \"e2e-derive-9f07f5b1-openshift-default-cd6f4f995-lxxs7\" (UID: \"755e8837-714b-455b-b3f6-0d03464aa561\") " pod="openshift-ingress/e2e-derive-9f07f5b1-openshift-default-cd6f4f995-lxxs7"
Jul 06 18:44:03.099811 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:44:03.099563    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/755e8837-714b-455b-b3f6-0d03464aa561-credential-socket\") pod \"e2e-derive-9f07f5b1-openshift-default-cd6f4f995-lxxs7\" (UID: \"755e8837-714b-455b-b3f6-0d03464aa561\") " pod="openshift-ingress/e2e-derive-9f07f5b1-openshift-default-cd6f4f995-lxxs7"
Jul 06 18:44:03.099811 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:44:03.099728    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/755e8837-714b-455b-b3f6-0d03464aa561-workload-socket\") pod \"e2e-derive-9f07f5b1-openshift-default-cd6f4f995-lxxs7\" (UID: \"755e8837-714b-455b-b3f6-0d03464aa561\") " pod="openshift-ingress/e2e-derive-9f07f5b1-openshift-default-cd6f4f995-lxxs7"
Jul 06 18:44:03.099811 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:44:03.099777    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/755e8837-714b-455b-b3f6-0d03464aa561-workload-certs\") pod \"e2e-derive-9f07f5b1-openshift-default-cd6f4f995-lxxs7\" (UID: \"755e8837-714b-455b-b3f6-0d03464aa561\") " pod="openshift-ingress/e2e-derive-9f07f5b1-openshift-default-cd6f4f995-lxxs7"
Jul 06 18:44:03.100147 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:44:03.100123    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/755e8837-714b-455b-b3f6-0d03464aa561-istio-envoy\") pod \"e2e-derive-9f07f5b1-openshift-default-cd6f4f995-lxxs7\" (UID: \"755e8837-714b-455b-b3f6-0d03464aa561\") " pod="openshift-ingress/e2e-derive-9f07f5b1-openshift-default-cd6f4f995-lxxs7"
Jul 06 18:44:03.100244 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:44:03.100153    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/755e8837-714b-455b-b3f6-0d03464aa561-istiod-ca-cert\") pod \"e2e-derive-9f07f5b1-openshift-default-cd6f4f995-lxxs7\" (UID: \"755e8837-714b-455b-b3f6-0d03464aa561\") " pod="openshift-ingress/e2e-derive-9f07f5b1-openshift-default-cd6f4f995-lxxs7"
Jul 06 18:44:03.100310 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:44:03.100282    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/755e8837-714b-455b-b3f6-0d03464aa561-istio-data\") pod \"e2e-derive-9f07f5b1-openshift-default-cd6f4f995-lxxs7\" (UID: \"755e8837-714b-455b-b3f6-0d03464aa561\") " pod="openshift-ingress/e2e-derive-9f07f5b1-openshift-default-cd6f4f995-lxxs7"
Jul 06 18:44:03.100405 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:44:03.100386    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/755e8837-714b-455b-b3f6-0d03464aa561-credential-socket\") pod \"e2e-derive-9f07f5b1-openshift-default-cd6f4f995-lxxs7\" (UID: \"755e8837-714b-455b-b3f6-0d03464aa561\") " pod="openshift-ingress/e2e-derive-9f07f5b1-openshift-default-cd6f4f995-lxxs7"
Jul 06 18:44:03.102089 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:44:03.102066    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/755e8837-714b-455b-b3f6-0d03464aa561-istio-podinfo\") pod \"e2e-derive-9f07f5b1-openshift-default-cd6f4f995-lxxs7\" (UID: \"755e8837-714b-455b-b3f6-0d03464aa561\") " pod="openshift-ingress/e2e-derive-9f07f5b1-openshift-default-cd6f4f995-lxxs7"
Jul 06 18:44:03.102273 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:44:03.102255    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/755e8837-714b-455b-b3f6-0d03464aa561-istio-envoy\") pod \"e2e-derive-9f07f5b1-openshift-default-cd6f4f995-lxxs7\" (UID: \"755e8837-714b-455b-b3f6-0d03464aa561\") " pod="openshift-ingress/e2e-derive-9f07f5b1-openshift-default-cd6f4f995-lxxs7"
Jul 06 18:44:03.108308 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:44:03.108283    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/755e8837-714b-455b-b3f6-0d03464aa561-istio-token\") pod \"e2e-derive-9f07f5b1-openshift-default-cd6f4f995-lxxs7\" (UID: \"755e8837-714b-455b-b3f6-0d03464aa561\") " pod="openshift-ingress/e2e-derive-9f07f5b1-openshift-default-cd6f4f995-lxxs7"
Jul 06 18:44:03.108394 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:44:03.108350    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5gp6w\" (UniqueName: \"kubernetes.io/projected/755e8837-714b-455b-b3f6-0d03464aa561-kube-api-access-5gp6w\") pod \"e2e-derive-9f07f5b1-openshift-default-cd6f4f995-lxxs7\" (UID: \"755e8837-714b-455b-b3f6-0d03464aa561\") " pod="openshift-ingress/e2e-derive-9f07f5b1-openshift-default-cd6f4f995-lxxs7"
Jul 06 18:44:03.249900 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:44:03.249801    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-9f07f5b1-openshift-default-cd6f4f995-lxxs7"
Jul 06 18:44:03.396568 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:44:03.396540    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-9f07f5b1-openshift-default-cd6f4f995-lxxs7"]
Jul 06 18:44:03.398922 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:44:03.398894    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod755e8837_714b_455b_b3f6_0d03464aa561.slice/crio-ca218ec65e3349edc71571eec1a13cf572c6126e4ef74e0c50480f8be10535f0 WatchSource:0}: Error finding container ca218ec65e3349edc71571eec1a13cf572c6126e4ef74e0c50480f8be10535f0: Status 404 returned error can't find the container with id ca218ec65e3349edc71571eec1a13cf572c6126e4ef74e0c50480f8be10535f0
Jul 06 18:44:03.401084 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:44:03.401052    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 06 18:44:03.401166 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:44:03.401117    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 06 18:44:03.401166 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:44:03.401144    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 06 18:44:04.193485 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:44:04.193444    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-derive-9f07f5b1-5bc7cfc9d8-4t8lj"]
Jul 06 18:44:04.195981 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:44:04.195959    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-9f07f5b1-5bc7cfc9d8-4t8lj"
Jul 06 18:44:04.199003 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:44:04.198982    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-derive-9f07f5b1-serving-cert\""
Jul 06 18:44:04.209415 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:44:04.209389    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-derive-9f07f5b1-5bc7cfc9d8-4t8lj"]
Jul 06 18:44:04.312992 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:44:04.312953    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b0282d70-a5e1-41ae-91f3-28c0c1f5de2b-maas-api-tls\") pod \"maas-api-e2e-derive-9f07f5b1-5bc7cfc9d8-4t8lj\" (UID: \"b0282d70-a5e1-41ae-91f3-28c0c1f5de2b\") " pod="opendatahub/maas-api-e2e-derive-9f07f5b1-5bc7cfc9d8-4t8lj"
Jul 06 18:44:04.312992 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:44:04.312989    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b0282d70-a5e1-41ae-91f3-28c0c1f5de2b-openshift-service-ca\") pod \"maas-api-e2e-derive-9f07f5b1-5bc7cfc9d8-4t8lj\" (UID: \"b0282d70-a5e1-41ae-91f3-28c0c1f5de2b\") " pod="opendatahub/maas-api-e2e-derive-9f07f5b1-5bc7cfc9d8-4t8lj"
Jul 06 18:44:04.313242 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:44:04.313088    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gs9rv\" (UniqueName: \"kubernetes.io/projected/b0282d70-a5e1-41ae-91f3-28c0c1f5de2b-kube-api-access-gs9rv\") pod \"maas-api-e2e-derive-9f07f5b1-5bc7cfc9d8-4t8lj\" (UID: \"b0282d70-a5e1-41ae-91f3-28c0c1f5de2b\") " pod="opendatahub/maas-api-e2e-derive-9f07f5b1-5bc7cfc9d8-4t8lj"
Jul 06 18:44:04.340553 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:44:04.340506    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-9f07f5b1-openshift-default-cd6f4f995-lxxs7" event={"ID":"755e8837-714b-455b-b3f6-0d03464aa561","Type":"ContainerStarted","Data":"f046827ee62857c51ea79561f5282563107f6ed3474b817491be2669070ce7e9"}
Jul 06 18:44:04.340721 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:44:04.340584    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-9f07f5b1-openshift-default-cd6f4f995-lxxs7" event={"ID":"755e8837-714b-455b-b3f6-0d03464aa561","Type":"ContainerStarted","Data":"ca218ec65e3349edc71571eec1a13cf572c6126e4ef74e0c50480f8be10535f0"}
Jul 06 18:44:04.394277 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:44:04.394211    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-9f07f5b1-openshift-default-cd6f4f995-lxxs7" podStartSLOduration=2.39418637 podStartE2EDuration="2.39418637s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:44:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:44:04.393879394 +0000 UTC m=+1579.551277883" watchObservedRunningTime="2026-07-06 18:44:04.39418637 +0000 UTC m=+1579.551584851"
Jul 06 18:44:04.414281 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:44:04.414234    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b0282d70-a5e1-41ae-91f3-28c0c1f5de2b-maas-api-tls\") pod \"maas-api-e2e-derive-9f07f5b1-5bc7cfc9d8-4t8lj\" (UID: \"b0282d70-a5e1-41ae-91f3-28c0c1f5de2b\") " pod="opendatahub/maas-api-e2e-derive-9f07f5b1-5bc7cfc9d8-4t8lj"
Jul 06 18:44:04.414281 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:44:04.414283    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b0282d70-a5e1-41ae-91f3-28c0c1f5de2b-openshift-service-ca\") pod \"maas-api-e2e-derive-9f07f5b1-5bc7cfc9d8-4t8lj\" (UID: \"b0282d70-a5e1-41ae-91f3-28c0c1f5de2b\") " pod="opendatahub/maas-api-e2e-derive-9f07f5b1-5bc7cfc9d8-4t8lj"
Jul 06 18:44:04.414595 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:44:04.414343    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gs9rv\" (UniqueName: \"kubernetes.io/projected/b0282d70-a5e1-41ae-91f3-28c0c1f5de2b-kube-api-access-gs9rv\") pod \"maas-api-e2e-derive-9f07f5b1-5bc7cfc9d8-4t8lj\" (UID: \"b0282d70-a5e1-41ae-91f3-28c0c1f5de2b\") " pod="opendatahub/maas-api-e2e-derive-9f07f5b1-5bc7cfc9d8-4t8lj"
Jul 06 18:44:04.415092 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:44:04.415062    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b0282d70-a5e1-41ae-91f3-28c0c1f5de2b-openshift-service-ca\") pod \"maas-api-e2e-derive-9f07f5b1-5bc7cfc9d8-4t8lj\" (UID: \"b0282d70-a5e1-41ae-91f3-28c0c1f5de2b\") " pod="opendatahub/maas-api-e2e-derive-9f07f5b1-5bc7cfc9d8-4t8lj"
Jul 06 18:44:04.416937 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:44:04.416920    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b0282d70-a5e1-41ae-91f3-28c0c1f5de2b-maas-api-tls\") pod \"maas-api-e2e-derive-9f07f5b1-5bc7cfc9d8-4t8lj\" (UID: \"b0282d70-a5e1-41ae-91f3-28c0c1f5de2b\") " pod="opendatahub/maas-api-e2e-derive-9f07f5b1-5bc7cfc9d8-4t8lj"
Jul 06 18:44:04.432769 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:44:04.432735    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gs9rv\" (UniqueName: \"kubernetes.io/projected/b0282d70-a5e1-41ae-91f3-28c0c1f5de2b-kube-api-access-gs9rv\") pod \"maas-api-e2e-derive-9f07f5b1-5bc7cfc9d8-4t8lj\" (UID: \"b0282d70-a5e1-41ae-91f3-28c0c1f5de2b\") " pod="opendatahub/maas-api-e2e-derive-9f07f5b1-5bc7cfc9d8-4t8lj"
Jul 06 18:44:04.507043 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:44:04.506867    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-9f07f5b1-5bc7cfc9d8-4t8lj"
Jul 06 18:44:04.651585 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:44:04.651557    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-derive-9f07f5b1-5bc7cfc9d8-4t8lj"]
Jul 06 18:44:04.653829 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:44:04.653799    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb0282d70_a5e1_41ae_91f3_28c0c1f5de2b.slice/crio-bf572368e69fc355fe1759b6a011bcc0b7a1c727f8fcaa4c490cdf38a580dcc8 WatchSource:0}: Error finding container bf572368e69fc355fe1759b6a011bcc0b7a1c727f8fcaa4c490cdf38a580dcc8: Status 404 returned error can't find the container with id bf572368e69fc355fe1759b6a011bcc0b7a1c727f8fcaa4c490cdf38a580dcc8
Jul 06 18:44:05.250729 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:44:05.250690    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-9f07f5b1-openshift-default-cd6f4f995-lxxs7"
Jul 06 18:44:05.255765 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:44:05.255738    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-9f07f5b1-openshift-default-cd6f4f995-lxxs7"
Jul 06 18:44:05.347942 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:44:05.347900    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-9f07f5b1-5bc7cfc9d8-4t8lj" event={"ID":"b0282d70-a5e1-41ae-91f3-28c0c1f5de2b","Type":"ContainerStarted","Data":"ccd9aaa4bb28adef28fd8f437bdf45f26b86c37d42459c0a6d492426a5a5a1d5"}
Jul 06 18:44:05.348173 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:44:05.347969    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-9f07f5b1-5bc7cfc9d8-4t8lj" event={"ID":"b0282d70-a5e1-41ae-91f3-28c0c1f5de2b","Type":"ContainerStarted","Data":"bf572368e69fc355fe1759b6a011bcc0b7a1c727f8fcaa4c490cdf38a580dcc8"}
Jul 06 18:44:05.348249 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:44:05.348196    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-derive-9f07f5b1-5bc7cfc9d8-4t8lj"
Jul 06 18:44:05.348249 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:44:05.348231    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-9f07f5b1-openshift-default-cd6f4f995-lxxs7"
Jul 06 18:44:05.349189 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:44:05.349168    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-9f07f5b1-openshift-default-cd6f4f995-lxxs7"
Jul 06 18:44:05.368704 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:44:05.368600    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-derive-9f07f5b1-5bc7cfc9d8-4t8lj" podStartSLOduration=0.904593828 podStartE2EDuration="1.368581367s" totalImagesPullingTime="463.987539ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:44:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:44:05.366647826 +0000 UTC m=+1580.524046313" watchObservedRunningTime="2026-07-06 18:44:05.368581367 +0000 UTC m=+1580.525979853"
Jul 06 18:44:08.705958 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:44:08.705914    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-9f07f5b1-openshift-default-cd6f4f995-lxxs7"]
Jul 06 18:44:08.706417 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:44:08.706167    2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-9f07f5b1-openshift-default-cd6f4f995-lxxs7" podUID="755e8837-714b-455b-b3f6-0d03464aa561" containerName="istio-proxy" containerID="cri-o://f046827ee62857c51ea79561f5282563107f6ed3474b817491be2669070ce7e9" gracePeriod=30
Jul 06 18:44:08.933567 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:44:08.933528    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-derive-9f07f5b1-5bc7cfc9d8-4t8lj"]
Jul 06 18:44:08.934162 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:44:08.934105    2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-derive-9f07f5b1-5bc7cfc9d8-4t8lj" podUID="b0282d70-a5e1-41ae-91f3-28c0c1f5de2b" containerName="maas-api" containerID="cri-o://ccd9aaa4bb28adef28fd8f437bdf45f26b86c37d42459c0a6d492426a5a5a1d5" gracePeriod=30
Jul 06 18:44:09.211161 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:44:09.211136    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-9f07f5b1-5bc7cfc9d8-4t8lj"
Jul 06 18:44:09.367641 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:44:09.367598    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b0282d70-a5e1-41ae-91f3-28c0c1f5de2b-maas-api-tls\") pod \"b0282d70-a5e1-41ae-91f3-28c0c1f5de2b\" (UID: \"b0282d70-a5e1-41ae-91f3-28c0c1f5de2b\") "
Jul 06 18:44:09.367839 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:44:09.367667    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b0282d70-a5e1-41ae-91f3-28c0c1f5de2b-openshift-service-ca\") pod \"b0282d70-a5e1-41ae-91f3-28c0c1f5de2b\" (UID: \"b0282d70-a5e1-41ae-91f3-28c0c1f5de2b\") "
Jul 06 18:44:09.367839 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:44:09.367754    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gs9rv\" (UniqueName: \"kubernetes.io/projected/b0282d70-a5e1-41ae-91f3-28c0c1f5de2b-kube-api-access-gs9rv\") pod \"b0282d70-a5e1-41ae-91f3-28c0c1f5de2b\" (UID: \"b0282d70-a5e1-41ae-91f3-28c0c1f5de2b\") "
Jul 06 18:44:09.368003 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:44:09.367977    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b0282d70-a5e1-41ae-91f3-28c0c1f5de2b-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b0282d70-a5e1-41ae-91f3-28c0c1f5de2b" (UID: "b0282d70-a5e1-41ae-91f3-28c0c1f5de2b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 18:44:09.369765 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:44:09.369739    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b0282d70-a5e1-41ae-91f3-28c0c1f5de2b-kube-api-access-gs9rv" (OuterVolumeSpecName: "kube-api-access-gs9rv") pod "b0282d70-a5e1-41ae-91f3-28c0c1f5de2b" (UID: "b0282d70-a5e1-41ae-91f3-28c0c1f5de2b"). InnerVolumeSpecName "kube-api-access-gs9rv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 18:44:09.369856 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:44:09.369835    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b0282d70-a5e1-41ae-91f3-28c0c1f5de2b-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b0282d70-a5e1-41ae-91f3-28c0c1f5de2b" (UID: "b0282d70-a5e1-41ae-91f3-28c0c1f5de2b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 18:44:09.374430 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:44:09.374404    2580 generic.go:356] "Generic (PLEG): container finished" podID="b0282d70-a5e1-41ae-91f3-28c0c1f5de2b" containerID="ccd9aaa4bb28adef28fd8f437bdf45f26b86c37d42459c0a6d492426a5a5a1d5" exitCode=0
Jul 06 18:44:09.374522 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:44:09.374447    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-9f07f5b1-5bc7cfc9d8-4t8lj"
Jul 06 18:44:09.374522 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:44:09.374490    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-9f07f5b1-5bc7cfc9d8-4t8lj" event={"ID":"b0282d70-a5e1-41ae-91f3-28c0c1f5de2b","Type":"ContainerDied","Data":"ccd9aaa4bb28adef28fd8f437bdf45f26b86c37d42459c0a6d492426a5a5a1d5"}
Jul 06 18:44:09.374649 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:44:09.374549    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-9f07f5b1-5bc7cfc9d8-4t8lj" event={"ID":"b0282d70-a5e1-41ae-91f3-28c0c1f5de2b","Type":"ContainerDied","Data":"bf572368e69fc355fe1759b6a011bcc0b7a1c727f8fcaa4c490cdf38a580dcc8"}
Jul 06 18:44:09.374649 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:44:09.374582    2580 scope.go:117] "RemoveContainer" containerID="ccd9aaa4bb28adef28fd8f437bdf45f26b86c37d42459c0a6d492426a5a5a1d5"
Jul 06 18:44:09.388795 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:44:09.388621    2580 scope.go:117] "RemoveContainer" containerID="ccd9aaa4bb28adef28fd8f437bdf45f26b86c37d42459c0a6d492426a5a5a1d5"
Jul 06 18:44:09.389152 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:44:09.389130    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ccd9aaa4bb28adef28fd8f437bdf45f26b86c37d42459c0a6d492426a5a5a1d5\": container with ID starting with ccd9aaa4bb28adef28fd8f437bdf45f26b86c37d42459c0a6d492426a5a5a1d5 not found: ID does not exist" containerID="ccd9aaa4bb28adef28fd8f437bdf45f26b86c37d42459c0a6d492426a5a5a1d5"
Jul 06 18:44:09.389250 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:44:09.389163    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ccd9aaa4bb28adef28fd8f437bdf45f26b86c37d42459c0a6d492426a5a5a1d5"} err="failed to get container status \"ccd9aaa4bb28adef28fd8f437bdf45f26b86c37d42459c0a6d492426a5a5a1d5\": rpc error: code = NotFound desc = could not find container \"ccd9aaa4bb28adef28fd8f437bdf45f26b86c37d42459c0a6d492426a5a5a1d5\": container with ID starting with ccd9aaa4bb28adef28fd8f437bdf45f26b86c37d42459c0a6d492426a5a5a1d5 not found: ID does not exist"
Jul 06 18:44:09.399266 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:44:09.399240    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-derive-9f07f5b1-5bc7cfc9d8-4t8lj"]
Jul 06 18:44:09.403663 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:44:09.403641    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-derive-9f07f5b1-5bc7cfc9d8-4t8lj"]
Jul 06 18:44:09.469502 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:44:09.469462    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b0282d70-a5e1-41ae-91f3-28c0c1f5de2b-openshift-service-ca\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:44:09.469762 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:44:09.469737    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gs9rv\" (UniqueName: \"kubernetes.io/projected/b0282d70-a5e1-41ae-91f3-28c0c1f5de2b-kube-api-access-gs9rv\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:44:09.469762 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:44:09.469761    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b0282d70-a5e1-41ae-91f3-28c0c1f5de2b-maas-api-tls\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:44:11.323601 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:44:11.323569    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b0282d70-a5e1-41ae-91f3-28c0c1f5de2b" path="/var/lib/kubelet/pods/b0282d70-a5e1-41ae-91f3-28c0c1f5de2b/volumes"
Jul 06 18:44:14.403976 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:44:14.403891    2580 generic.go:356] "Generic (PLEG): container finished" podID="755e8837-714b-455b-b3f6-0d03464aa561" containerID="f046827ee62857c51ea79561f5282563107f6ed3474b817491be2669070ce7e9" exitCode=0
Jul 06 18:44:14.404491 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:44:14.403986    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-9f07f5b1-openshift-default-cd6f4f995-lxxs7" event={"ID":"755e8837-714b-455b-b3f6-0d03464aa561","Type":"ContainerDied","Data":"f046827ee62857c51ea79561f5282563107f6ed3474b817491be2669070ce7e9"}
Jul 06 18:44:14.462965 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:44:14.462943    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-9f07f5b1-openshift-default-cd6f4f995-lxxs7"
Jul 06 18:44:14.615724 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:44:14.615690    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/755e8837-714b-455b-b3f6-0d03464aa561-istiod-ca-cert\") pod \"755e8837-714b-455b-b3f6-0d03464aa561\" (UID: \"755e8837-714b-455b-b3f6-0d03464aa561\") "
Jul 06 18:44:14.615919 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:44:14.615774    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/755e8837-714b-455b-b3f6-0d03464aa561-istio-token\") pod \"755e8837-714b-455b-b3f6-0d03464aa561\" (UID: \"755e8837-714b-455b-b3f6-0d03464aa561\") "
Jul 06 18:44:14.615919 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:44:14.615811    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/755e8837-714b-455b-b3f6-0d03464aa561-istio-podinfo\") pod \"755e8837-714b-455b-b3f6-0d03464aa561\" (UID: \"755e8837-714b-455b-b3f6-0d03464aa561\") "
Jul 06 18:44:14.615919 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:44:14.615841    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/755e8837-714b-455b-b3f6-0d03464aa561-istio-data\") pod \"755e8837-714b-455b-b3f6-0d03464aa561\" (UID: \"755e8837-714b-455b-b3f6-0d03464aa561\") "
Jul 06 18:44:14.616123 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:44:14.616047    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/755e8837-714b-455b-b3f6-0d03464aa561-istio-envoy\") pod \"755e8837-714b-455b-b3f6-0d03464aa561\" (UID: \"755e8837-714b-455b-b3f6-0d03464aa561\") "
Jul 06 18:44:14.616123 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:44:14.616102    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/755e8837-714b-455b-b3f6-0d03464aa561-workload-socket\") pod \"755e8837-714b-455b-b3f6-0d03464aa561\" (UID: \"755e8837-714b-455b-b3f6-0d03464aa561\") "
Jul 06 18:44:14.616237 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:44:14.616142    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5gp6w\" (UniqueName: \"kubernetes.io/projected/755e8837-714b-455b-b3f6-0d03464aa561-kube-api-access-5gp6w\") pod \"755e8837-714b-455b-b3f6-0d03464aa561\" (UID: \"755e8837-714b-455b-b3f6-0d03464aa561\") "
Jul 06 18:44:14.616237 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:44:14.616182    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/755e8837-714b-455b-b3f6-0d03464aa561-credential-socket\") pod \"755e8837-714b-455b-b3f6-0d03464aa561\" (UID: \"755e8837-714b-455b-b3f6-0d03464aa561\") "
Jul 06 18:44:14.616237 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:44:14.616215    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/755e8837-714b-455b-b3f6-0d03464aa561-workload-certs\") pod \"755e8837-714b-455b-b3f6-0d03464aa561\" (UID: \"755e8837-714b-455b-b3f6-0d03464aa561\") "
Jul 06 18:44:14.616541 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:44:14.616211    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/755e8837-714b-455b-b3f6-0d03464aa561-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "755e8837-714b-455b-b3f6-0d03464aa561" (UID: "755e8837-714b-455b-b3f6-0d03464aa561"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 18:44:14.616541 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:44:14.616319    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/755e8837-714b-455b-b3f6-0d03464aa561-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "755e8837-714b-455b-b3f6-0d03464aa561" (UID: "755e8837-714b-455b-b3f6-0d03464aa561"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 18:44:14.616541 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:44:14.616411    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/755e8837-714b-455b-b3f6-0d03464aa561-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "755e8837-714b-455b-b3f6-0d03464aa561" (UID: "755e8837-714b-455b-b3f6-0d03464aa561"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 18:44:14.616541 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:44:14.616518    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/755e8837-714b-455b-b3f6-0d03464aa561-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "755e8837-714b-455b-b3f6-0d03464aa561" (UID: "755e8837-714b-455b-b3f6-0d03464aa561"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 18:44:14.616760 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:44:14.616609    2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/755e8837-714b-455b-b3f6-0d03464aa561-workload-socket\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:44:14.616760 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:44:14.616632    2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/755e8837-714b-455b-b3f6-0d03464aa561-credential-socket\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:44:14.616760 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:44:14.616647    2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/755e8837-714b-455b-b3f6-0d03464aa561-workload-certs\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:44:14.616760 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:44:14.616660    2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/755e8837-714b-455b-b3f6-0d03464aa561-istiod-ca-cert\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:44:14.616929 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:44:14.616760    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/755e8837-714b-455b-b3f6-0d03464aa561-istio-data" (OuterVolumeSpecName: "istio-data") pod "755e8837-714b-455b-b3f6-0d03464aa561" (UID: "755e8837-714b-455b-b3f6-0d03464aa561"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 18:44:14.618207 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:44:14.618186    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/755e8837-714b-455b-b3f6-0d03464aa561-istio-token" (OuterVolumeSpecName: "istio-token") pod "755e8837-714b-455b-b3f6-0d03464aa561" (UID: "755e8837-714b-455b-b3f6-0d03464aa561"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 18:44:14.618344 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:44:14.618226    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/755e8837-714b-455b-b3f6-0d03464aa561-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "755e8837-714b-455b-b3f6-0d03464aa561" (UID: "755e8837-714b-455b-b3f6-0d03464aa561"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 18:44:14.618593 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:44:14.618577    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/755e8837-714b-455b-b3f6-0d03464aa561-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "755e8837-714b-455b-b3f6-0d03464aa561" (UID: "755e8837-714b-455b-b3f6-0d03464aa561"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 06 18:44:14.618655 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:44:14.618581    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/755e8837-714b-455b-b3f6-0d03464aa561-kube-api-access-5gp6w" (OuterVolumeSpecName: "kube-api-access-5gp6w") pod "755e8837-714b-455b-b3f6-0d03464aa561" (UID: "755e8837-714b-455b-b3f6-0d03464aa561"). InnerVolumeSpecName "kube-api-access-5gp6w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 18:44:14.717660 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:44:14.717622    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5gp6w\" (UniqueName: \"kubernetes.io/projected/755e8837-714b-455b-b3f6-0d03464aa561-kube-api-access-5gp6w\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:44:14.717660 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:44:14.717654    2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/755e8837-714b-455b-b3f6-0d03464aa561-istio-token\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:44:14.717660 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:44:14.717663    2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/755e8837-714b-455b-b3f6-0d03464aa561-istio-podinfo\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:44:14.717897 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:44:14.717671    2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/755e8837-714b-455b-b3f6-0d03464aa561-istio-data\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:44:14.717897 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:44:14.717680    2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/755e8837-714b-455b-b3f6-0d03464aa561-istio-envoy\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:44:15.409859 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:44:15.409822    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-9f07f5b1-openshift-default-cd6f4f995-lxxs7" event={"ID":"755e8837-714b-455b-b3f6-0d03464aa561","Type":"ContainerDied","Data":"ca218ec65e3349edc71571eec1a13cf572c6126e4ef74e0c50480f8be10535f0"}
Jul 06 18:44:15.409859 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:44:15.409848    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-9f07f5b1-openshift-default-cd6f4f995-lxxs7"
Jul 06 18:44:15.410383 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:44:15.409879    2580 scope.go:117] "RemoveContainer" containerID="f046827ee62857c51ea79561f5282563107f6ed3474b817491be2669070ce7e9"
Jul 06 18:44:15.436964 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:44:15.436940    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-9f07f5b1-openshift-default-cd6f4f995-lxxs7"]
Jul 06 18:44:15.447404 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:44:15.447379    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-9f07f5b1-openshift-default-cd6f4f995-lxxs7"]
Jul 06 18:44:17.323558 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:44:17.323525    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="755e8837-714b-455b-b3f6-0d03464aa561" path="/var/lib/kubelet/pods/755e8837-714b-455b-b3f6-0d03464aa561/volumes"
Jul 06 18:45:00.135188 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:45:00.135152    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-29722725-dx9b7"]
Jul 06 18:45:00.135644 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:45:00.135525    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="755e8837-714b-455b-b3f6-0d03464aa561" containerName="istio-proxy"
Jul 06 18:45:00.135644 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:45:00.135536    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="755e8837-714b-455b-b3f6-0d03464aa561" containerName="istio-proxy"
Jul 06 18:45:00.135644 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:45:00.135543    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b0282d70-a5e1-41ae-91f3-28c0c1f5de2b" containerName="maas-api"
Jul 06 18:45:00.135644 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:45:00.135548    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="b0282d70-a5e1-41ae-91f3-28c0c1f5de2b" containerName="maas-api"
Jul 06 18:45:00.135644 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:45:00.135617    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="755e8837-714b-455b-b3f6-0d03464aa561" containerName="istio-proxy"
Jul 06 18:45:00.135644 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:45:00.135626    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="b0282d70-a5e1-41ae-91f3-28c0c1f5de2b" containerName="maas-api"
Jul 06 18:45:00.138550 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:45:00.138534    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29722725-dx9b7"
Jul 06 18:45:00.146674 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:45:00.146648    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29722725-dx9b7"]
Jul 06 18:45:00.302434 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:45:00.302378    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q4m2d\" (UniqueName: \"kubernetes.io/projected/17eeefb8-1ca4-4c82-b73f-fe031dbbd201-kube-api-access-q4m2d\") pod \"maas-api-key-cleanup-29722725-dx9b7\" (UID: \"17eeefb8-1ca4-4c82-b73f-fe031dbbd201\") " pod="opendatahub/maas-api-key-cleanup-29722725-dx9b7"
Jul 06 18:45:00.403503 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:45:00.403409    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q4m2d\" (UniqueName: \"kubernetes.io/projected/17eeefb8-1ca4-4c82-b73f-fe031dbbd201-kube-api-access-q4m2d\") pod \"maas-api-key-cleanup-29722725-dx9b7\" (UID: \"17eeefb8-1ca4-4c82-b73f-fe031dbbd201\") " pod="opendatahub/maas-api-key-cleanup-29722725-dx9b7"
Jul 06 18:45:00.412505 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:45:00.412474    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q4m2d\" (UniqueName: \"kubernetes.io/projected/17eeefb8-1ca4-4c82-b73f-fe031dbbd201-kube-api-access-q4m2d\") pod \"maas-api-key-cleanup-29722725-dx9b7\" (UID: \"17eeefb8-1ca4-4c82-b73f-fe031dbbd201\") " pod="opendatahub/maas-api-key-cleanup-29722725-dx9b7"
Jul 06 18:45:00.449301 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:45:00.449267    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29722725-dx9b7"
Jul 06 18:45:00.578308 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:45:00.578281    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29722725-dx9b7"]
Jul 06 18:45:00.580439 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:45:00.580403    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17eeefb8_1ca4_4c82_b73f_fe031dbbd201.slice/crio-61e8f788ddcf2dc153bfb58c297742b1f405c26805170cffddaf7c4daa8eb34f WatchSource:0}: Error finding container 61e8f788ddcf2dc153bfb58c297742b1f405c26805170cffddaf7c4daa8eb34f: Status 404 returned error can't find the container with id 61e8f788ddcf2dc153bfb58c297742b1f405c26805170cffddaf7c4daa8eb34f
Jul 06 18:45:00.620086 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:45:00.620057    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29722725-dx9b7" event={"ID":"17eeefb8-1ca4-4c82-b73f-fe031dbbd201","Type":"ContainerStarted","Data":"61e8f788ddcf2dc153bfb58c297742b1f405c26805170cffddaf7c4daa8eb34f"}
Jul 06 18:45:01.625762 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:45:01.625726    2580 generic.go:356] "Generic (PLEG): container finished" podID="17eeefb8-1ca4-4c82-b73f-fe031dbbd201" containerID="ef60b34aff65a376effd7ebe34ee6395c08753de3f6fceb796e5b0f9ba50e1c3" exitCode=0
Jul 06 18:45:01.626185 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:45:01.625804    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29722725-dx9b7" event={"ID":"17eeefb8-1ca4-4c82-b73f-fe031dbbd201","Type":"ContainerDied","Data":"ef60b34aff65a376effd7ebe34ee6395c08753de3f6fceb796e5b0f9ba50e1c3"}
Jul 06 18:45:02.765171 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:45:02.765146    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29722725-dx9b7"
Jul 06 18:45:02.925983 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:45:02.925879    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q4m2d\" (UniqueName: \"kubernetes.io/projected/17eeefb8-1ca4-4c82-b73f-fe031dbbd201-kube-api-access-q4m2d\") pod \"17eeefb8-1ca4-4c82-b73f-fe031dbbd201\" (UID: \"17eeefb8-1ca4-4c82-b73f-fe031dbbd201\") "
Jul 06 18:45:02.928078 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:45:02.928049    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/17eeefb8-1ca4-4c82-b73f-fe031dbbd201-kube-api-access-q4m2d" (OuterVolumeSpecName: "kube-api-access-q4m2d") pod "17eeefb8-1ca4-4c82-b73f-fe031dbbd201" (UID: "17eeefb8-1ca4-4c82-b73f-fe031dbbd201"). InnerVolumeSpecName "kube-api-access-q4m2d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 18:45:03.027386 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:45:03.027345    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q4m2d\" (UniqueName: \"kubernetes.io/projected/17eeefb8-1ca4-4c82-b73f-fe031dbbd201-kube-api-access-q4m2d\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:45:03.636529 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:45:03.636448    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29722725-dx9b7"
Jul 06 18:45:03.636529 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:45:03.636463    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29722725-dx9b7" event={"ID":"17eeefb8-1ca4-4c82-b73f-fe031dbbd201","Type":"ContainerDied","Data":"61e8f788ddcf2dc153bfb58c297742b1f405c26805170cffddaf7c4daa8eb34f"}
Jul 06 18:45:03.636529 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:45:03.636506    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="61e8f788ddcf2dc153bfb58c297742b1f405c26805170cffddaf7c4daa8eb34f"
Jul 06 18:45:51.178481 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:45:51.178447    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-9379b309-openshift-default-f667b468c-v9n6v"]
Jul 06 18:45:51.178935 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:45:51.178863    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="17eeefb8-1ca4-4c82-b73f-fe031dbbd201" containerName="cleanup"
Jul 06 18:45:51.178935 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:45:51.178876    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="17eeefb8-1ca4-4c82-b73f-fe031dbbd201" containerName="cleanup"
Jul 06 18:45:51.179007 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:45:51.178950    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="17eeefb8-1ca4-4c82-b73f-fe031dbbd201" containerName="cleanup"
Jul 06 18:45:51.181969 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:45:51.181948    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-9379b309-openshift-default-f667b468c-v9n6v"
Jul 06 18:45:51.185366 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:45:51.185343    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-9379b309-openshift-default-dockercfg-dfrkc\""
Jul 06 18:45:51.197606 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:45:51.197582    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-9379b309-openshift-default-f667b468c-v9n6v"]
Jul 06 18:45:51.260401 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:45:51.260367    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a19fc6cf-31cd-47e5-81bc-77ef9140cc7f-credential-socket\") pod \"e2e-mt-9379b309-openshift-default-f667b468c-v9n6v\" (UID: \"a19fc6cf-31cd-47e5-81bc-77ef9140cc7f\") " pod="openshift-ingress/e2e-mt-9379b309-openshift-default-f667b468c-v9n6v"
Jul 06 18:45:51.260590 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:45:51.260427    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a19fc6cf-31cd-47e5-81bc-77ef9140cc7f-istio-podinfo\") pod \"e2e-mt-9379b309-openshift-default-f667b468c-v9n6v\" (UID: \"a19fc6cf-31cd-47e5-81bc-77ef9140cc7f\") " pod="openshift-ingress/e2e-mt-9379b309-openshift-default-f667b468c-v9n6v"
Jul 06 18:45:51.260590 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:45:51.260456    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a19fc6cf-31cd-47e5-81bc-77ef9140cc7f-workload-socket\") pod \"e2e-mt-9379b309-openshift-default-f667b468c-v9n6v\" (UID: \"a19fc6cf-31cd-47e5-81bc-77ef9140cc7f\") " pod="openshift-ingress/e2e-mt-9379b309-openshift-default-f667b468c-v9n6v"
Jul 06 18:45:51.260590 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:45:51.260479    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a19fc6cf-31cd-47e5-81bc-77ef9140cc7f-workload-certs\") pod \"e2e-mt-9379b309-openshift-default-f667b468c-v9n6v\" (UID: \"a19fc6cf-31cd-47e5-81bc-77ef9140cc7f\") " pod="openshift-ingress/e2e-mt-9379b309-openshift-default-f667b468c-v9n6v"
Jul 06 18:45:51.260590 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:45:51.260502    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a19fc6cf-31cd-47e5-81bc-77ef9140cc7f-istio-token\") pod \"e2e-mt-9379b309-openshift-default-f667b468c-v9n6v\" (UID: \"a19fc6cf-31cd-47e5-81bc-77ef9140cc7f\") " pod="openshift-ingress/e2e-mt-9379b309-openshift-default-f667b468c-v9n6v"
Jul 06 18:45:51.260590 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:45:51.260575    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a19fc6cf-31cd-47e5-81bc-77ef9140cc7f-istio-envoy\") pod \"e2e-mt-9379b309-openshift-default-f667b468c-v9n6v\" (UID: \"a19fc6cf-31cd-47e5-81bc-77ef9140cc7f\") " pod="openshift-ingress/e2e-mt-9379b309-openshift-default-f667b468c-v9n6v"
Jul 06 18:45:51.260813 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:45:51.260631    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a19fc6cf-31cd-47e5-81bc-77ef9140cc7f-istiod-ca-cert\") pod \"e2e-mt-9379b309-openshift-default-f667b468c-v9n6v\" (UID: \"a19fc6cf-31cd-47e5-81bc-77ef9140cc7f\") " pod="openshift-ingress/e2e-mt-9379b309-openshift-default-f667b468c-v9n6v"
Jul 06 18:45:51.260813 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:45:51.260673    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a19fc6cf-31cd-47e5-81bc-77ef9140cc7f-istio-data\") pod \"e2e-mt-9379b309-openshift-default-f667b468c-v9n6v\" (UID: \"a19fc6cf-31cd-47e5-81bc-77ef9140cc7f\") " pod="openshift-ingress/e2e-mt-9379b309-openshift-default-f667b468c-v9n6v"
Jul 06 18:45:51.260813 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:45:51.260742    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qqfhw\" (UniqueName: \"kubernetes.io/projected/a19fc6cf-31cd-47e5-81bc-77ef9140cc7f-kube-api-access-qqfhw\") pod \"e2e-mt-9379b309-openshift-default-f667b468c-v9n6v\" (UID: \"a19fc6cf-31cd-47e5-81bc-77ef9140cc7f\") " pod="openshift-ingress/e2e-mt-9379b309-openshift-default-f667b468c-v9n6v"
Jul 06 18:45:51.361551 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:45:51.361512    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qqfhw\" (UniqueName: \"kubernetes.io/projected/a19fc6cf-31cd-47e5-81bc-77ef9140cc7f-kube-api-access-qqfhw\") pod \"e2e-mt-9379b309-openshift-default-f667b468c-v9n6v\" (UID: \"a19fc6cf-31cd-47e5-81bc-77ef9140cc7f\") " pod="openshift-ingress/e2e-mt-9379b309-openshift-default-f667b468c-v9n6v"
Jul 06 18:45:51.361723 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:45:51.361578    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a19fc6cf-31cd-47e5-81bc-77ef9140cc7f-credential-socket\") pod \"e2e-mt-9379b309-openshift-default-f667b468c-v9n6v\" (UID: \"a19fc6cf-31cd-47e5-81bc-77ef9140cc7f\") " pod="openshift-ingress/e2e-mt-9379b309-openshift-default-f667b468c-v9n6v"
Jul 06 18:45:51.361723 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:45:51.361608    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a19fc6cf-31cd-47e5-81bc-77ef9140cc7f-istio-podinfo\") pod \"e2e-mt-9379b309-openshift-default-f667b468c-v9n6v\" (UID: \"a19fc6cf-31cd-47e5-81bc-77ef9140cc7f\") " pod="openshift-ingress/e2e-mt-9379b309-openshift-default-f667b468c-v9n6v"
Jul 06 18:45:51.361723 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:45:51.361624    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a19fc6cf-31cd-47e5-81bc-77ef9140cc7f-workload-socket\") pod \"e2e-mt-9379b309-openshift-default-f667b468c-v9n6v\" (UID: \"a19fc6cf-31cd-47e5-81bc-77ef9140cc7f\") " pod="openshift-ingress/e2e-mt-9379b309-openshift-default-f667b468c-v9n6v"
Jul 06 18:45:51.361723 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:45:51.361649    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a19fc6cf-31cd-47e5-81bc-77ef9140cc7f-workload-certs\") pod \"e2e-mt-9379b309-openshift-default-f667b468c-v9n6v\" (UID: \"a19fc6cf-31cd-47e5-81bc-77ef9140cc7f\") " pod="openshift-ingress/e2e-mt-9379b309-openshift-default-f667b468c-v9n6v"
Jul 06 18:45:51.361940 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:45:51.361718    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a19fc6cf-31cd-47e5-81bc-77ef9140cc7f-istio-token\") pod \"e2e-mt-9379b309-openshift-default-f667b468c-v9n6v\" (UID: \"a19fc6cf-31cd-47e5-81bc-77ef9140cc7f\") " pod="openshift-ingress/e2e-mt-9379b309-openshift-default-f667b468c-v9n6v"
Jul 06 18:45:51.361940 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:45:51.361769    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a19fc6cf-31cd-47e5-81bc-77ef9140cc7f-istio-envoy\") pod \"e2e-mt-9379b309-openshift-default-f667b468c-v9n6v\" (UID: \"a19fc6cf-31cd-47e5-81bc-77ef9140cc7f\") " pod="openshift-ingress/e2e-mt-9379b309-openshift-default-f667b468c-v9n6v"
Jul 06 18:45:51.361940 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:45:51.361852    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a19fc6cf-31cd-47e5-81bc-77ef9140cc7f-istiod-ca-cert\") pod \"e2e-mt-9379b309-openshift-default-f667b468c-v9n6v\" (UID: \"a19fc6cf-31cd-47e5-81bc-77ef9140cc7f\") " pod="openshift-ingress/e2e-mt-9379b309-openshift-default-f667b468c-v9n6v"
Jul 06 18:45:51.361940 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:45:51.361915    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a19fc6cf-31cd-47e5-81bc-77ef9140cc7f-istio-data\") pod \"e2e-mt-9379b309-openshift-default-f667b468c-v9n6v\" (UID: \"a19fc6cf-31cd-47e5-81bc-77ef9140cc7f\") " pod="openshift-ingress/e2e-mt-9379b309-openshift-default-f667b468c-v9n6v"
Jul 06 18:45:51.362147 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:45:51.362064    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a19fc6cf-31cd-47e5-81bc-77ef9140cc7f-workload-certs\") pod \"e2e-mt-9379b309-openshift-default-f667b468c-v9n6v\" (UID: \"a19fc6cf-31cd-47e5-81bc-77ef9140cc7f\") " pod="openshift-ingress/e2e-mt-9379b309-openshift-default-f667b468c-v9n6v"
Jul 06 18:45:51.362256 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:45:51.362232    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a19fc6cf-31cd-47e5-81bc-77ef9140cc7f-workload-socket\") pod \"e2e-mt-9379b309-openshift-default-f667b468c-v9n6v\" (UID: \"a19fc6cf-31cd-47e5-81bc-77ef9140cc7f\") " pod="openshift-ingress/e2e-mt-9379b309-openshift-default-f667b468c-v9n6v"
Jul 06 18:45:51.362326 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:45:51.362287    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a19fc6cf-31cd-47e5-81bc-77ef9140cc7f-credential-socket\") pod \"e2e-mt-9379b309-openshift-default-f667b468c-v9n6v\" (UID: \"a19fc6cf-31cd-47e5-81bc-77ef9140cc7f\") " pod="openshift-ingress/e2e-mt-9379b309-openshift-default-f667b468c-v9n6v"
Jul 06 18:45:51.362404 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:45:51.362384    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a19fc6cf-31cd-47e5-81bc-77ef9140cc7f-istio-data\") pod \"e2e-mt-9379b309-openshift-default-f667b468c-v9n6v\" (UID: \"a19fc6cf-31cd-47e5-81bc-77ef9140cc7f\") " pod="openshift-ingress/e2e-mt-9379b309-openshift-default-f667b468c-v9n6v"
Jul 06 18:45:51.362620 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:45:51.362599    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a19fc6cf-31cd-47e5-81bc-77ef9140cc7f-istiod-ca-cert\") pod \"e2e-mt-9379b309-openshift-default-f667b468c-v9n6v\" (UID: \"a19fc6cf-31cd-47e5-81bc-77ef9140cc7f\") " pod="openshift-ingress/e2e-mt-9379b309-openshift-default-f667b468c-v9n6v"
Jul 06 18:45:51.364103 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:45:51.364079    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a19fc6cf-31cd-47e5-81bc-77ef9140cc7f-istio-envoy\") pod \"e2e-mt-9379b309-openshift-default-f667b468c-v9n6v\" (UID: \"a19fc6cf-31cd-47e5-81bc-77ef9140cc7f\") " pod="openshift-ingress/e2e-mt-9379b309-openshift-default-f667b468c-v9n6v"
Jul 06 18:45:51.364222 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:45:51.364205    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a19fc6cf-31cd-47e5-81bc-77ef9140cc7f-istio-podinfo\") pod \"e2e-mt-9379b309-openshift-default-f667b468c-v9n6v\" (UID: \"a19fc6cf-31cd-47e5-81bc-77ef9140cc7f\") " pod="openshift-ingress/e2e-mt-9379b309-openshift-default-f667b468c-v9n6v"
Jul 06 18:45:51.369431 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:45:51.369410    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a19fc6cf-31cd-47e5-81bc-77ef9140cc7f-istio-token\") pod \"e2e-mt-9379b309-openshift-default-f667b468c-v9n6v\" (UID: \"a19fc6cf-31cd-47e5-81bc-77ef9140cc7f\") " pod="openshift-ingress/e2e-mt-9379b309-openshift-default-f667b468c-v9n6v"
Jul 06 18:45:51.369690 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:45:51.369669    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qqfhw\" (UniqueName: \"kubernetes.io/projected/a19fc6cf-31cd-47e5-81bc-77ef9140cc7f-kube-api-access-qqfhw\") pod \"e2e-mt-9379b309-openshift-default-f667b468c-v9n6v\" (UID: \"a19fc6cf-31cd-47e5-81bc-77ef9140cc7f\") " pod="openshift-ingress/e2e-mt-9379b309-openshift-default-f667b468c-v9n6v"
Jul 06 18:45:51.492058 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:45:51.491947    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-9379b309-openshift-default-f667b468c-v9n6v"
Jul 06 18:45:51.634644 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:45:51.634620    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-9379b309-openshift-default-f667b468c-v9n6v"]
Jul 06 18:45:51.637097 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:45:51.637063    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda19fc6cf_31cd_47e5_81bc_77ef9140cc7f.slice/crio-3da1671226526a067c4d9c6610a75cc305d5c29ec685d36b26df0324ba09456c WatchSource:0}: Error finding container 3da1671226526a067c4d9c6610a75cc305d5c29ec685d36b26df0324ba09456c: Status 404 returned error can't find the container with id 3da1671226526a067c4d9c6610a75cc305d5c29ec685d36b26df0324ba09456c
Jul 06 18:45:51.639167 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:45:51.639130    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 06 18:45:51.639243 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:45:51.639219    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 06 18:45:51.639293 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:45:51.639262    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 06 18:45:51.876204 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:45:51.876170    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-9379b309-openshift-default-f667b468c-v9n6v" event={"ID":"a19fc6cf-31cd-47e5-81bc-77ef9140cc7f","Type":"ContainerStarted","Data":"2c18354be9368d2d856a2315303fc2c7cdd92a257c2c2a12ebcb86cfcfa436cc"}
Jul 06 18:45:51.876377 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:45:51.876227    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-9379b309-openshift-default-f667b468c-v9n6v" event={"ID":"a19fc6cf-31cd-47e5-81bc-77ef9140cc7f","Type":"ContainerStarted","Data":"3da1671226526a067c4d9c6610a75cc305d5c29ec685d36b26df0324ba09456c"}
Jul 06 18:45:51.900181 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:45:51.900121    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-9379b309-openshift-default-f667b468c-v9n6v" podStartSLOduration=0.900102831 podStartE2EDuration="900.102831ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:45:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:45:51.897601843 +0000 UTC m=+1687.055000328" watchObservedRunningTime="2026-07-06 18:45:51.900102831 +0000 UTC m=+1687.057501321"
Jul 06 18:45:52.492395 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:45:52.492360    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-9379b309-openshift-default-f667b468c-v9n6v"
Jul 06 18:45:52.497273 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:45:52.497250    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-9379b309-openshift-default-f667b468c-v9n6v"
Jul 06 18:45:52.881929 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:45:52.881901    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-9379b309-openshift-default-f667b468c-v9n6v"
Jul 06 18:45:52.882895 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:45:52.882870    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-9379b309-openshift-default-f667b468c-v9n6v"
Jul 06 18:45:54.151685 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:45:54.151652    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-mt-9379b309-64459dccd4-qjg97"]
Jul 06 18:45:54.155483 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:45:54.155454    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-9379b309-64459dccd4-qjg97"
Jul 06 18:45:54.159569 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:45:54.159541    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-mt-9379b309-serving-cert\""
Jul 06 18:45:54.170605 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:45:54.170579    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-9379b309-64459dccd4-qjg97"]
Jul 06 18:45:54.285383 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:45:54.285346    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gjtxc\" (UniqueName: \"kubernetes.io/projected/0cb59266-dad2-429f-9728-22eefd4b40ac-kube-api-access-gjtxc\") pod \"maas-api-e2e-mt-9379b309-64459dccd4-qjg97\" (UID: \"0cb59266-dad2-429f-9728-22eefd4b40ac\") " pod="opendatahub/maas-api-e2e-mt-9379b309-64459dccd4-qjg97"
Jul 06 18:45:54.285383 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:45:54.285381    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0cb59266-dad2-429f-9728-22eefd4b40ac-openshift-service-ca\") pod \"maas-api-e2e-mt-9379b309-64459dccd4-qjg97\" (UID: \"0cb59266-dad2-429f-9728-22eefd4b40ac\") " pod="opendatahub/maas-api-e2e-mt-9379b309-64459dccd4-qjg97"
Jul 06 18:45:54.285642 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:45:54.285505    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0cb59266-dad2-429f-9728-22eefd4b40ac-maas-api-tls\") pod \"maas-api-e2e-mt-9379b309-64459dccd4-qjg97\" (UID: \"0cb59266-dad2-429f-9728-22eefd4b40ac\") " pod="opendatahub/maas-api-e2e-mt-9379b309-64459dccd4-qjg97"
Jul 06 18:45:54.386617 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:45:54.386579    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0cb59266-dad2-429f-9728-22eefd4b40ac-maas-api-tls\") pod \"maas-api-e2e-mt-9379b309-64459dccd4-qjg97\" (UID: \"0cb59266-dad2-429f-9728-22eefd4b40ac\") " pod="opendatahub/maas-api-e2e-mt-9379b309-64459dccd4-qjg97"
Jul 06 18:45:54.386828 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:45:54.386644    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gjtxc\" (UniqueName: \"kubernetes.io/projected/0cb59266-dad2-429f-9728-22eefd4b40ac-kube-api-access-gjtxc\") pod \"maas-api-e2e-mt-9379b309-64459dccd4-qjg97\" (UID: \"0cb59266-dad2-429f-9728-22eefd4b40ac\") " pod="opendatahub/maas-api-e2e-mt-9379b309-64459dccd4-qjg97"
Jul 06 18:45:54.386828 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:45:54.386663    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0cb59266-dad2-429f-9728-22eefd4b40ac-openshift-service-ca\") pod \"maas-api-e2e-mt-9379b309-64459dccd4-qjg97\" (UID: \"0cb59266-dad2-429f-9728-22eefd4b40ac\") " pod="opendatahub/maas-api-e2e-mt-9379b309-64459dccd4-qjg97"
Jul 06 18:45:54.386828 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:45:54.386736    2580 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-mt-9379b309-serving-cert: secret "maas-api-e2e-mt-9379b309-serving-cert" not found
Jul 06 18:45:54.386828 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:45:54.386801    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0cb59266-dad2-429f-9728-22eefd4b40ac-maas-api-tls podName:0cb59266-dad2-429f-9728-22eefd4b40ac nodeName:}" failed. No retries permitted until 2026-07-06 18:45:54.886783732 +0000 UTC m=+1690.044182200 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/0cb59266-dad2-429f-9728-22eefd4b40ac-maas-api-tls") pod "maas-api-e2e-mt-9379b309-64459dccd4-qjg97" (UID: "0cb59266-dad2-429f-9728-22eefd4b40ac") : secret "maas-api-e2e-mt-9379b309-serving-cert" not found
Jul 06 18:45:54.387350 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:45:54.387331    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0cb59266-dad2-429f-9728-22eefd4b40ac-openshift-service-ca\") pod \"maas-api-e2e-mt-9379b309-64459dccd4-qjg97\" (UID: \"0cb59266-dad2-429f-9728-22eefd4b40ac\") " pod="opendatahub/maas-api-e2e-mt-9379b309-64459dccd4-qjg97"
Jul 06 18:45:54.399576 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:45:54.399551    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gjtxc\" (UniqueName: \"kubernetes.io/projected/0cb59266-dad2-429f-9728-22eefd4b40ac-kube-api-access-gjtxc\") pod \"maas-api-e2e-mt-9379b309-64459dccd4-qjg97\" (UID: \"0cb59266-dad2-429f-9728-22eefd4b40ac\") " pod="opendatahub/maas-api-e2e-mt-9379b309-64459dccd4-qjg97"
Jul 06 18:45:54.891303 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:45:54.891167    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0cb59266-dad2-429f-9728-22eefd4b40ac-maas-api-tls\") pod \"maas-api-e2e-mt-9379b309-64459dccd4-qjg97\" (UID: \"0cb59266-dad2-429f-9728-22eefd4b40ac\") " pod="opendatahub/maas-api-e2e-mt-9379b309-64459dccd4-qjg97"
Jul 06 18:45:54.894349 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:45:54.894309    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0cb59266-dad2-429f-9728-22eefd4b40ac-maas-api-tls\") pod \"maas-api-e2e-mt-9379b309-64459dccd4-qjg97\" (UID: \"0cb59266-dad2-429f-9728-22eefd4b40ac\") " pod="opendatahub/maas-api-e2e-mt-9379b309-64459dccd4-qjg97"
Jul 06 18:45:55.068131 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:45:55.068097    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-9379b309-64459dccd4-qjg97"
Jul 06 18:45:55.216576 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:45:55.216551    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-9379b309-64459dccd4-qjg97"]
Jul 06 18:45:55.219307 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:45:55.219279    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0cb59266_dad2_429f_9728_22eefd4b40ac.slice/crio-08819c51a6dca16f5d3ca09eafff0de849702d22de249ac9d2b550515fb37eb8 WatchSource:0}: Error finding container 08819c51a6dca16f5d3ca09eafff0de849702d22de249ac9d2b550515fb37eb8: Status 404 returned error can't find the container with id 08819c51a6dca16f5d3ca09eafff0de849702d22de249ac9d2b550515fb37eb8
Jul 06 18:45:55.898421 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:45:55.898389    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-9379b309-64459dccd4-qjg97" event={"ID":"0cb59266-dad2-429f-9728-22eefd4b40ac","Type":"ContainerStarted","Data":"0b960b376b97aaf33b9d44c978e86fe290ff84756168039519ee3964efaf3b33"}
Jul 06 18:45:55.898578 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:45:55.898441    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-9379b309-64459dccd4-qjg97" event={"ID":"0cb59266-dad2-429f-9728-22eefd4b40ac","Type":"ContainerStarted","Data":"08819c51a6dca16f5d3ca09eafff0de849702d22de249ac9d2b550515fb37eb8"}
Jul 06 18:45:55.898578 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:45:55.898463    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-mt-9379b309-64459dccd4-qjg97"
Jul 06 18:45:55.921595 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:45:55.921542    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-mt-9379b309-64459dccd4-qjg97" podStartSLOduration=1.319562324 podStartE2EDuration="1.921527572s" totalImagesPullingTime="601.965248ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:45:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:45:55.921151969 +0000 UTC m=+1691.078550467" watchObservedRunningTime="2026-07-06 18:45:55.921527572 +0000 UTC m=+1691.078926050"
Jul 06 18:46:01.908010 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:01.907981    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-mt-9379b309-64459dccd4-qjg97"
Jul 06 18:46:06.177251 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:06.177205    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-9379b309-64459dccd4-qjg97"]
Jul 06 18:46:06.177725 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:06.177453    2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-mt-9379b309-64459dccd4-qjg97" podUID="0cb59266-dad2-429f-9728-22eefd4b40ac" containerName="maas-api" containerID="cri-o://0b960b376b97aaf33b9d44c978e86fe290ff84756168039519ee3964efaf3b33" gracePeriod=30
Jul 06 18:46:06.444574 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:06.444509    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-9379b309-64459dccd4-qjg97"
Jul 06 18:46:06.497157 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:06.497126    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gjtxc\" (UniqueName: \"kubernetes.io/projected/0cb59266-dad2-429f-9728-22eefd4b40ac-kube-api-access-gjtxc\") pod \"0cb59266-dad2-429f-9728-22eefd4b40ac\" (UID: \"0cb59266-dad2-429f-9728-22eefd4b40ac\") "
Jul 06 18:46:06.497157 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:06.497161    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0cb59266-dad2-429f-9728-22eefd4b40ac-openshift-service-ca\") pod \"0cb59266-dad2-429f-9728-22eefd4b40ac\" (UID: \"0cb59266-dad2-429f-9728-22eefd4b40ac\") "
Jul 06 18:46:06.497444 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:06.497183    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0cb59266-dad2-429f-9728-22eefd4b40ac-maas-api-tls\") pod \"0cb59266-dad2-429f-9728-22eefd4b40ac\" (UID: \"0cb59266-dad2-429f-9728-22eefd4b40ac\") "
Jul 06 18:46:06.497636 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:06.497604    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0cb59266-dad2-429f-9728-22eefd4b40ac-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "0cb59266-dad2-429f-9728-22eefd4b40ac" (UID: "0cb59266-dad2-429f-9728-22eefd4b40ac"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 18:46:06.499541 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:06.499510    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0cb59266-dad2-429f-9728-22eefd4b40ac-kube-api-access-gjtxc" (OuterVolumeSpecName: "kube-api-access-gjtxc") pod "0cb59266-dad2-429f-9728-22eefd4b40ac" (UID: "0cb59266-dad2-429f-9728-22eefd4b40ac"). InnerVolumeSpecName "kube-api-access-gjtxc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 18:46:06.499806 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:06.499787    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0cb59266-dad2-429f-9728-22eefd4b40ac-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "0cb59266-dad2-429f-9728-22eefd4b40ac" (UID: "0cb59266-dad2-429f-9728-22eefd4b40ac"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 18:46:06.597809 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:06.597770    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gjtxc\" (UniqueName: \"kubernetes.io/projected/0cb59266-dad2-429f-9728-22eefd4b40ac-kube-api-access-gjtxc\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:46:06.597809 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:06.597802    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0cb59266-dad2-429f-9728-22eefd4b40ac-openshift-service-ca\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:46:06.597809 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:06.597814    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0cb59266-dad2-429f-9728-22eefd4b40ac-maas-api-tls\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:46:06.952649 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:06.952557    2580 generic.go:356] "Generic (PLEG): container finished" podID="a19fc6cf-31cd-47e5-81bc-77ef9140cc7f" containerID="2c18354be9368d2d856a2315303fc2c7cdd92a257c2c2a12ebcb86cfcfa436cc" exitCode=0
Jul 06 18:46:06.952649 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:06.952631    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-9379b309-openshift-default-f667b468c-v9n6v" event={"ID":"a19fc6cf-31cd-47e5-81bc-77ef9140cc7f","Type":"ContainerDied","Data":"2c18354be9368d2d856a2315303fc2c7cdd92a257c2c2a12ebcb86cfcfa436cc"}
Jul 06 18:46:06.953171 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:06.953107    2580 scope.go:117] "RemoveContainer" containerID="2c18354be9368d2d856a2315303fc2c7cdd92a257c2c2a12ebcb86cfcfa436cc"
Jul 06 18:46:06.954088 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:06.954068    2580 generic.go:356] "Generic (PLEG): container finished" podID="0cb59266-dad2-429f-9728-22eefd4b40ac" containerID="0b960b376b97aaf33b9d44c978e86fe290ff84756168039519ee3964efaf3b33" exitCode=0
Jul 06 18:46:06.954196 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:06.954104    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-9379b309-64459dccd4-qjg97" event={"ID":"0cb59266-dad2-429f-9728-22eefd4b40ac","Type":"ContainerDied","Data":"0b960b376b97aaf33b9d44c978e86fe290ff84756168039519ee3964efaf3b33"}
Jul 06 18:46:06.954196 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:06.954125    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-9379b309-64459dccd4-qjg97"
Jul 06 18:46:06.954196 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:06.954135    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-9379b309-64459dccd4-qjg97" event={"ID":"0cb59266-dad2-429f-9728-22eefd4b40ac","Type":"ContainerDied","Data":"08819c51a6dca16f5d3ca09eafff0de849702d22de249ac9d2b550515fb37eb8"}
Jul 06 18:46:06.954196 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:06.954155    2580 scope.go:117] "RemoveContainer" containerID="0b960b376b97aaf33b9d44c978e86fe290ff84756168039519ee3964efaf3b33"
Jul 06 18:46:06.954196 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:06.954182    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 06 18:46:06.954420 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:06.954262    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 06 18:46:06.954420 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:06.954311    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 06 18:46:06.966836 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:06.966815    2580 scope.go:117] "RemoveContainer" containerID="0b960b376b97aaf33b9d44c978e86fe290ff84756168039519ee3964efaf3b33"
Jul 06 18:46:06.967140 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:46:06.967118    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0b960b376b97aaf33b9d44c978e86fe290ff84756168039519ee3964efaf3b33\": container with ID starting with 0b960b376b97aaf33b9d44c978e86fe290ff84756168039519ee3964efaf3b33 not found: ID does not exist" containerID="0b960b376b97aaf33b9d44c978e86fe290ff84756168039519ee3964efaf3b33"
Jul 06 18:46:06.967215 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:06.967147    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0b960b376b97aaf33b9d44c978e86fe290ff84756168039519ee3964efaf3b33"} err="failed to get container status \"0b960b376b97aaf33b9d44c978e86fe290ff84756168039519ee3964efaf3b33\": rpc error: code = NotFound desc = could not find container \"0b960b376b97aaf33b9d44c978e86fe290ff84756168039519ee3964efaf3b33\": container with ID starting with 0b960b376b97aaf33b9d44c978e86fe290ff84756168039519ee3964efaf3b33 not found: ID does not exist"
Jul 06 18:46:06.993762 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:06.993732    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-9379b309-64459dccd4-qjg97"]
Jul 06 18:46:06.998226 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:06.998194    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-mt-9379b309-64459dccd4-qjg97"]
Jul 06 18:46:07.323190 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:07.323165    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0cb59266-dad2-429f-9728-22eefd4b40ac" path="/var/lib/kubelet/pods/0cb59266-dad2-429f-9728-22eefd4b40ac/volumes"
Jul 06 18:46:07.882652 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:07.882612    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-9379b309-openshift-default-f667b468c-v9n6v"
Jul 06 18:46:07.960480 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:07.960439    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-9379b309-openshift-default-f667b468c-v9n6v" event={"ID":"a19fc6cf-31cd-47e5-81bc-77ef9140cc7f","Type":"ContainerStarted","Data":"f1135e02f993c1e701e2690ad307b50e0b7f7612c45765b335145636e8dd1a82"}
Jul 06 18:46:08.492249 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:08.492213    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-9379b309-openshift-default-f667b468c-v9n6v"
Jul 06 18:46:08.493393 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:08.493367    2580 patch_prober.go:28] interesting pod/e2e-mt-9379b309-openshift-default-f667b468c-v9n6v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body=
Jul 06 18:46:08.493507 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:08.493418    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-9379b309-openshift-default-f667b468c-v9n6v" podUID="a19fc6cf-31cd-47e5-81bc-77ef9140cc7f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused"
Jul 06 18:46:09.493276 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:09.493225    2580 patch_prober.go:28] interesting pod/e2e-mt-9379b309-openshift-default-f667b468c-v9n6v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body=
Jul 06 18:46:09.493641 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:09.493336    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-9379b309-openshift-default-f667b468c-v9n6v" podUID="a19fc6cf-31cd-47e5-81bc-77ef9140cc7f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused"
Jul 06 18:46:10.492644 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:10.492595    2580 patch_prober.go:28] interesting pod/e2e-mt-9379b309-openshift-default-f667b468c-v9n6v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body=
Jul 06 18:46:10.492824 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:10.492683    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-9379b309-openshift-default-f667b468c-v9n6v" podUID="a19fc6cf-31cd-47e5-81bc-77ef9140cc7f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused"
Jul 06 18:46:11.492886 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:11.492847    2580 patch_prober.go:28] interesting pod/e2e-mt-9379b309-openshift-default-f667b468c-v9n6v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body=
Jul 06 18:46:11.493332 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:11.492916    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-9379b309-openshift-default-f667b468c-v9n6v" podUID="a19fc6cf-31cd-47e5-81bc-77ef9140cc7f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused"
Jul 06 18:46:12.328893 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:12.328856    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-9379b309-openshift-default-f667b468c-v9n6v"]
Jul 06 18:46:12.329132 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:12.329105    2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-9379b309-openshift-default-f667b468c-v9n6v" podUID="a19fc6cf-31cd-47e5-81bc-77ef9140cc7f" containerName="istio-proxy" containerID="cri-o://f1135e02f993c1e701e2690ad307b50e0b7f7612c45765b335145636e8dd1a82" gracePeriod=30
Jul 06 18:46:17.593412 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:17.593345    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-9379b309-openshift-default-f667b468c-v9n6v"
Jul 06 18:46:17.688223 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:17.688189    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a19fc6cf-31cd-47e5-81bc-77ef9140cc7f-workload-certs\") pod \"a19fc6cf-31cd-47e5-81bc-77ef9140cc7f\" (UID: \"a19fc6cf-31cd-47e5-81bc-77ef9140cc7f\") "
Jul 06 18:46:17.688223 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:17.688228    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qqfhw\" (UniqueName: \"kubernetes.io/projected/a19fc6cf-31cd-47e5-81bc-77ef9140cc7f-kube-api-access-qqfhw\") pod \"a19fc6cf-31cd-47e5-81bc-77ef9140cc7f\" (UID: \"a19fc6cf-31cd-47e5-81bc-77ef9140cc7f\") "
Jul 06 18:46:17.688433 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:17.688303    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a19fc6cf-31cd-47e5-81bc-77ef9140cc7f-credential-socket\") pod \"a19fc6cf-31cd-47e5-81bc-77ef9140cc7f\" (UID: \"a19fc6cf-31cd-47e5-81bc-77ef9140cc7f\") "
Jul 06 18:46:17.688433 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:17.688345    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a19fc6cf-31cd-47e5-81bc-77ef9140cc7f-workload-socket\") pod \"a19fc6cf-31cd-47e5-81bc-77ef9140cc7f\" (UID: \"a19fc6cf-31cd-47e5-81bc-77ef9140cc7f\") "
Jul 06 18:46:17.688433 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:17.688399    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a19fc6cf-31cd-47e5-81bc-77ef9140cc7f-istiod-ca-cert\") pod \"a19fc6cf-31cd-47e5-81bc-77ef9140cc7f\" (UID: \"a19fc6cf-31cd-47e5-81bc-77ef9140cc7f\") "
Jul 06 18:46:17.688433 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:17.688423    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a19fc6cf-31cd-47e5-81bc-77ef9140cc7f-istio-data\") pod \"a19fc6cf-31cd-47e5-81bc-77ef9140cc7f\" (UID: \"a19fc6cf-31cd-47e5-81bc-77ef9140cc7f\") "
Jul 06 18:46:17.688636 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:17.688463    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a19fc6cf-31cd-47e5-81bc-77ef9140cc7f-istio-podinfo\") pod \"a19fc6cf-31cd-47e5-81bc-77ef9140cc7f\" (UID: \"a19fc6cf-31cd-47e5-81bc-77ef9140cc7f\") "
Jul 06 18:46:17.688636 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:17.688510    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a19fc6cf-31cd-47e5-81bc-77ef9140cc7f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a19fc6cf-31cd-47e5-81bc-77ef9140cc7f" (UID: "a19fc6cf-31cd-47e5-81bc-77ef9140cc7f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 18:46:17.688636 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:17.688519    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a19fc6cf-31cd-47e5-81bc-77ef9140cc7f-istio-envoy\") pod \"a19fc6cf-31cd-47e5-81bc-77ef9140cc7f\" (UID: \"a19fc6cf-31cd-47e5-81bc-77ef9140cc7f\") "
Jul 06 18:46:17.688636 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:17.688548    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a19fc6cf-31cd-47e5-81bc-77ef9140cc7f-istio-token\") pod \"a19fc6cf-31cd-47e5-81bc-77ef9140cc7f\" (UID: \"a19fc6cf-31cd-47e5-81bc-77ef9140cc7f\") "
Jul 06 18:46:17.688844 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:17.688624    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a19fc6cf-31cd-47e5-81bc-77ef9140cc7f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a19fc6cf-31cd-47e5-81bc-77ef9140cc7f" (UID: "a19fc6cf-31cd-47e5-81bc-77ef9140cc7f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 18:46:17.688902 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:17.688839    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a19fc6cf-31cd-47e5-81bc-77ef9140cc7f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a19fc6cf-31cd-47e5-81bc-77ef9140cc7f" (UID: "a19fc6cf-31cd-47e5-81bc-77ef9140cc7f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 18:46:17.688902 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:17.688876    2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a19fc6cf-31cd-47e5-81bc-77ef9140cc7f-credential-socket\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:46:17.688902 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:17.688898    2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a19fc6cf-31cd-47e5-81bc-77ef9140cc7f-workload-certs\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:46:17.689088 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:17.688943    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a19fc6cf-31cd-47e5-81bc-77ef9140cc7f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a19fc6cf-31cd-47e5-81bc-77ef9140cc7f" (UID: "a19fc6cf-31cd-47e5-81bc-77ef9140cc7f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 18:46:17.689419 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:17.689393    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a19fc6cf-31cd-47e5-81bc-77ef9140cc7f-istio-data" (OuterVolumeSpecName: "istio-data") pod "a19fc6cf-31cd-47e5-81bc-77ef9140cc7f" (UID: "a19fc6cf-31cd-47e5-81bc-77ef9140cc7f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 18:46:17.690591 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:17.690544    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a19fc6cf-31cd-47e5-81bc-77ef9140cc7f-kube-api-access-qqfhw" (OuterVolumeSpecName: "kube-api-access-qqfhw") pod "a19fc6cf-31cd-47e5-81bc-77ef9140cc7f" (UID: "a19fc6cf-31cd-47e5-81bc-77ef9140cc7f"). InnerVolumeSpecName "kube-api-access-qqfhw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 18:46:17.690722 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:17.690658    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a19fc6cf-31cd-47e5-81bc-77ef9140cc7f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a19fc6cf-31cd-47e5-81bc-77ef9140cc7f" (UID: "a19fc6cf-31cd-47e5-81bc-77ef9140cc7f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 18:46:17.690971 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:17.690953    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a19fc6cf-31cd-47e5-81bc-77ef9140cc7f-istio-token" (OuterVolumeSpecName: "istio-token") pod "a19fc6cf-31cd-47e5-81bc-77ef9140cc7f" (UID: "a19fc6cf-31cd-47e5-81bc-77ef9140cc7f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 18:46:17.691037 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:17.690965    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a19fc6cf-31cd-47e5-81bc-77ef9140cc7f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a19fc6cf-31cd-47e5-81bc-77ef9140cc7f" (UID: "a19fc6cf-31cd-47e5-81bc-77ef9140cc7f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 06 18:46:17.789741 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:17.789699    2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a19fc6cf-31cd-47e5-81bc-77ef9140cc7f-workload-socket\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:46:17.789741 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:17.789734    2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a19fc6cf-31cd-47e5-81bc-77ef9140cc7f-istiod-ca-cert\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:46:17.789741 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:17.789743    2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a19fc6cf-31cd-47e5-81bc-77ef9140cc7f-istio-data\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:46:17.789741 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:17.789751    2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a19fc6cf-31cd-47e5-81bc-77ef9140cc7f-istio-podinfo\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:46:17.790059 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:17.789759    2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a19fc6cf-31cd-47e5-81bc-77ef9140cc7f-istio-envoy\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:46:17.790059 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:17.789768    2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a19fc6cf-31cd-47e5-81bc-77ef9140cc7f-istio-token\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:46:17.790059 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:17.789776    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qqfhw\" (UniqueName: \"kubernetes.io/projected/a19fc6cf-31cd-47e5-81bc-77ef9140cc7f-kube-api-access-qqfhw\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:46:18.009887 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:18.009799    2580 generic.go:356] "Generic (PLEG): container finished" podID="a19fc6cf-31cd-47e5-81bc-77ef9140cc7f" containerID="f1135e02f993c1e701e2690ad307b50e0b7f7612c45765b335145636e8dd1a82" exitCode=0
Jul 06 18:46:18.009887 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:18.009851    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-9379b309-openshift-default-f667b468c-v9n6v"
Jul 06 18:46:18.009887 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:18.009873    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-9379b309-openshift-default-f667b468c-v9n6v" event={"ID":"a19fc6cf-31cd-47e5-81bc-77ef9140cc7f","Type":"ContainerDied","Data":"f1135e02f993c1e701e2690ad307b50e0b7f7612c45765b335145636e8dd1a82"}
Jul 06 18:46:18.010197 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:18.009927    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-9379b309-openshift-default-f667b468c-v9n6v" event={"ID":"a19fc6cf-31cd-47e5-81bc-77ef9140cc7f","Type":"ContainerDied","Data":"3da1671226526a067c4d9c6610a75cc305d5c29ec685d36b26df0324ba09456c"}
Jul 06 18:46:18.010197 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:18.009956    2580 scope.go:117] "RemoveContainer" containerID="f1135e02f993c1e701e2690ad307b50e0b7f7612c45765b335145636e8dd1a82"
Jul 06 18:46:18.019260 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:18.019242    2580 scope.go:117] "RemoveContainer" containerID="2c18354be9368d2d856a2315303fc2c7cdd92a257c2c2a12ebcb86cfcfa436cc"
Jul 06 18:46:18.026890 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:18.026871    2580 scope.go:117] "RemoveContainer" containerID="f1135e02f993c1e701e2690ad307b50e0b7f7612c45765b335145636e8dd1a82"
Jul 06 18:46:18.027255 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:46:18.027234    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f1135e02f993c1e701e2690ad307b50e0b7f7612c45765b335145636e8dd1a82\": container with ID starting with f1135e02f993c1e701e2690ad307b50e0b7f7612c45765b335145636e8dd1a82 not found: ID does not exist" containerID="f1135e02f993c1e701e2690ad307b50e0b7f7612c45765b335145636e8dd1a82"
Jul 06 18:46:18.027348 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:18.027266    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f1135e02f993c1e701e2690ad307b50e0b7f7612c45765b335145636e8dd1a82"} err="failed to get container status \"f1135e02f993c1e701e2690ad307b50e0b7f7612c45765b335145636e8dd1a82\": rpc error: code = NotFound desc = could not find container \"f1135e02f993c1e701e2690ad307b50e0b7f7612c45765b335145636e8dd1a82\": container with ID starting with f1135e02f993c1e701e2690ad307b50e0b7f7612c45765b335145636e8dd1a82 not found: ID does not exist"
Jul 06 18:46:18.027348 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:18.027289    2580 scope.go:117] "RemoveContainer" containerID="2c18354be9368d2d856a2315303fc2c7cdd92a257c2c2a12ebcb86cfcfa436cc"
Jul 06 18:46:18.027540 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:46:18.027523    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2c18354be9368d2d856a2315303fc2c7cdd92a257c2c2a12ebcb86cfcfa436cc\": container with ID starting with 2c18354be9368d2d856a2315303fc2c7cdd92a257c2c2a12ebcb86cfcfa436cc not found: ID does not exist" containerID="2c18354be9368d2d856a2315303fc2c7cdd92a257c2c2a12ebcb86cfcfa436cc"
Jul 06 18:46:18.027598 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:18.027547    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2c18354be9368d2d856a2315303fc2c7cdd92a257c2c2a12ebcb86cfcfa436cc"} err="failed to get container status \"2c18354be9368d2d856a2315303fc2c7cdd92a257c2c2a12ebcb86cfcfa436cc\": rpc error: code = NotFound desc = could not find container \"2c18354be9368d2d856a2315303fc2c7cdd92a257c2c2a12ebcb86cfcfa436cc\": container with ID starting with 2c18354be9368d2d856a2315303fc2c7cdd92a257c2c2a12ebcb86cfcfa436cc not found: ID does not exist"
Jul 06 18:46:18.034553 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:18.034531    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-9379b309-openshift-default-f667b468c-v9n6v"]
Jul 06 18:46:18.036403 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:18.036382    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-9379b309-openshift-default-f667b468c-v9n6v"]
Jul 06 18:46:19.322783 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:19.322749    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a19fc6cf-31cd-47e5-81bc-77ef9140cc7f" path="/var/lib/kubelet/pods/a19fc6cf-31cd-47e5-81bc-77ef9140cc7f/volumes"
Jul 06 18:46:32.001183 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:32.001147    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-452bd22f-openshift-default-6c77575-g68ls"]
Jul 06 18:46:32.001716 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:32.001700    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0cb59266-dad2-429f-9728-22eefd4b40ac" containerName="maas-api"
Jul 06 18:46:32.001772 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:32.001719    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="0cb59266-dad2-429f-9728-22eefd4b40ac" containerName="maas-api"
Jul 06 18:46:32.001772 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:32.001734    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a19fc6cf-31cd-47e5-81bc-77ef9140cc7f" containerName="istio-proxy"
Jul 06 18:46:32.001772 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:32.001742    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="a19fc6cf-31cd-47e5-81bc-77ef9140cc7f" containerName="istio-proxy"
Jul 06 18:46:32.001772 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:32.001755    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a19fc6cf-31cd-47e5-81bc-77ef9140cc7f" containerName="istio-proxy"
Jul 06 18:46:32.001772 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:32.001763    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="a19fc6cf-31cd-47e5-81bc-77ef9140cc7f" containerName="istio-proxy"
Jul 06 18:46:32.001940 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:32.001846    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="a19fc6cf-31cd-47e5-81bc-77ef9140cc7f" containerName="istio-proxy"
Jul 06 18:46:32.001940 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:32.001861    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="0cb59266-dad2-429f-9728-22eefd4b40ac" containerName="maas-api"
Jul 06 18:46:32.006149 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:32.006128    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-452bd22f-openshift-default-6c77575-g68ls"
Jul 06 18:46:32.009816 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:32.009794    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-452bd22f-openshift-default-dockercfg-sk99j\""
Jul 06 18:46:32.020767 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:32.020732    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-452bd22f-openshift-default-6c77575-g68ls"]
Jul 06 18:46:32.116118 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:32.116084    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e20e860-569b-4a59-bd0d-c4bac25f9bd3-credential-socket\") pod \"e2e-mt-452bd22f-openshift-default-6c77575-g68ls\" (UID: \"5e20e860-569b-4a59-bd0d-c4bac25f9bd3\") " pod="openshift-ingress/e2e-mt-452bd22f-openshift-default-6c77575-g68ls"
Jul 06 18:46:32.116303 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:32.116142    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5e20e860-569b-4a59-bd0d-c4bac25f9bd3-istio-data\") pod \"e2e-mt-452bd22f-openshift-default-6c77575-g68ls\" (UID: \"5e20e860-569b-4a59-bd0d-c4bac25f9bd3\") " pod="openshift-ingress/e2e-mt-452bd22f-openshift-default-6c77575-g68ls"
Jul 06 18:46:32.116303 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:32.116201    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e20e860-569b-4a59-bd0d-c4bac25f9bd3-workload-socket\") pod \"e2e-mt-452bd22f-openshift-default-6c77575-g68ls\" (UID: \"5e20e860-569b-4a59-bd0d-c4bac25f9bd3\") " pod="openshift-ingress/e2e-mt-452bd22f-openshift-default-6c77575-g68ls"
Jul 06 18:46:32.116303 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:32.116242    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5e20e860-569b-4a59-bd0d-c4bac25f9bd3-istio-envoy\") pod \"e2e-mt-452bd22f-openshift-default-6c77575-g68ls\" (UID: \"5e20e860-569b-4a59-bd0d-c4bac25f9bd3\") " pod="openshift-ingress/e2e-mt-452bd22f-openshift-default-6c77575-g68ls"
Jul 06 18:46:32.116498 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:32.116310    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5e20e860-569b-4a59-bd0d-c4bac25f9bd3-istio-podinfo\") pod \"e2e-mt-452bd22f-openshift-default-6c77575-g68ls\" (UID: \"5e20e860-569b-4a59-bd0d-c4bac25f9bd3\") " pod="openshift-ingress/e2e-mt-452bd22f-openshift-default-6c77575-g68ls"
Jul 06 18:46:32.116498 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:32.116355    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5e20e860-569b-4a59-bd0d-c4bac25f9bd3-workload-certs\") pod \"e2e-mt-452bd22f-openshift-default-6c77575-g68ls\" (UID: \"5e20e860-569b-4a59-bd0d-c4bac25f9bd3\") " pod="openshift-ingress/e2e-mt-452bd22f-openshift-default-6c77575-g68ls"
Jul 06 18:46:32.116498 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:32.116383    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5e20e860-569b-4a59-bd0d-c4bac25f9bd3-istio-token\") pod \"e2e-mt-452bd22f-openshift-default-6c77575-g68ls\" (UID: \"5e20e860-569b-4a59-bd0d-c4bac25f9bd3\") " pod="openshift-ingress/e2e-mt-452bd22f-openshift-default-6c77575-g68ls"
Jul 06 18:46:32.116498 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:32.116410    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4ffft\" (UniqueName: \"kubernetes.io/projected/5e20e860-569b-4a59-bd0d-c4bac25f9bd3-kube-api-access-4ffft\") pod \"e2e-mt-452bd22f-openshift-default-6c77575-g68ls\" (UID: \"5e20e860-569b-4a59-bd0d-c4bac25f9bd3\") " pod="openshift-ingress/e2e-mt-452bd22f-openshift-default-6c77575-g68ls"
Jul 06 18:46:32.116498 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:32.116468    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5e20e860-569b-4a59-bd0d-c4bac25f9bd3-istiod-ca-cert\") pod \"e2e-mt-452bd22f-openshift-default-6c77575-g68ls\" (UID: \"5e20e860-569b-4a59-bd0d-c4bac25f9bd3\") " pod="openshift-ingress/e2e-mt-452bd22f-openshift-default-6c77575-g68ls"
Jul 06 18:46:32.217530 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:32.217495    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e20e860-569b-4a59-bd0d-c4bac25f9bd3-credential-socket\") pod \"e2e-mt-452bd22f-openshift-default-6c77575-g68ls\" (UID: \"5e20e860-569b-4a59-bd0d-c4bac25f9bd3\") " pod="openshift-ingress/e2e-mt-452bd22f-openshift-default-6c77575-g68ls"
Jul 06 18:46:32.217530 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:32.217542    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5e20e860-569b-4a59-bd0d-c4bac25f9bd3-istio-data\") pod \"e2e-mt-452bd22f-openshift-default-6c77575-g68ls\" (UID: \"5e20e860-569b-4a59-bd0d-c4bac25f9bd3\") " pod="openshift-ingress/e2e-mt-452bd22f-openshift-default-6c77575-g68ls"
Jul 06 18:46:32.217797 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:32.217563    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e20e860-569b-4a59-bd0d-c4bac25f9bd3-workload-socket\") pod \"e2e-mt-452bd22f-openshift-default-6c77575-g68ls\" (UID: \"5e20e860-569b-4a59-bd0d-c4bac25f9bd3\") " pod="openshift-ingress/e2e-mt-452bd22f-openshift-default-6c77575-g68ls"
Jul 06 18:46:32.217797 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:32.217579    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5e20e860-569b-4a59-bd0d-c4bac25f9bd3-istio-envoy\") pod \"e2e-mt-452bd22f-openshift-default-6c77575-g68ls\" (UID: \"5e20e860-569b-4a59-bd0d-c4bac25f9bd3\") " pod="openshift-ingress/e2e-mt-452bd22f-openshift-default-6c77575-g68ls"
Jul 06 18:46:32.217797 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:32.217612    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5e20e860-569b-4a59-bd0d-c4bac25f9bd3-istio-podinfo\") pod \"e2e-mt-452bd22f-openshift-default-6c77575-g68ls\" (UID: \"5e20e860-569b-4a59-bd0d-c4bac25f9bd3\") " pod="openshift-ingress/e2e-mt-452bd22f-openshift-default-6c77575-g68ls"
Jul 06 18:46:32.217797 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:32.217641    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5e20e860-569b-4a59-bd0d-c4bac25f9bd3-workload-certs\") pod \"e2e-mt-452bd22f-openshift-default-6c77575-g68ls\" (UID: \"5e20e860-569b-4a59-bd0d-c4bac25f9bd3\") " pod="openshift-ingress/e2e-mt-452bd22f-openshift-default-6c77575-g68ls"
Jul 06 18:46:32.217797 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:32.217663    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5e20e860-569b-4a59-bd0d-c4bac25f9bd3-istio-token\") pod \"e2e-mt-452bd22f-openshift-default-6c77575-g68ls\" (UID: \"5e20e860-569b-4a59-bd0d-c4bac25f9bd3\") " pod="openshift-ingress/e2e-mt-452bd22f-openshift-default-6c77575-g68ls"
Jul 06 18:46:32.217797 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:32.217689    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4ffft\" (UniqueName: \"kubernetes.io/projected/5e20e860-569b-4a59-bd0d-c4bac25f9bd3-kube-api-access-4ffft\") pod \"e2e-mt-452bd22f-openshift-default-6c77575-g68ls\" (UID: \"5e20e860-569b-4a59-bd0d-c4bac25f9bd3\") " pod="openshift-ingress/e2e-mt-452bd22f-openshift-default-6c77575-g68ls"
Jul 06 18:46:32.217797 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:32.217734    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5e20e860-569b-4a59-bd0d-c4bac25f9bd3-istiod-ca-cert\") pod \"e2e-mt-452bd22f-openshift-default-6c77575-g68ls\" (UID: \"5e20e860-569b-4a59-bd0d-c4bac25f9bd3\") " pod="openshift-ingress/e2e-mt-452bd22f-openshift-default-6c77575-g68ls"
Jul 06 18:46:32.218162 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:32.217959    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e20e860-569b-4a59-bd0d-c4bac25f9bd3-credential-socket\") pod \"e2e-mt-452bd22f-openshift-default-6c77575-g68ls\" (UID: \"5e20e860-569b-4a59-bd0d-c4bac25f9bd3\") " pod="openshift-ingress/e2e-mt-452bd22f-openshift-default-6c77575-g68ls"
Jul 06 18:46:32.218222 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:32.218176    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e20e860-569b-4a59-bd0d-c4bac25f9bd3-workload-socket\") pod \"e2e-mt-452bd22f-openshift-default-6c77575-g68ls\" (UID: \"5e20e860-569b-4a59-bd0d-c4bac25f9bd3\") " pod="openshift-ingress/e2e-mt-452bd22f-openshift-default-6c77575-g68ls"
Jul 06 18:46:32.218222 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:32.218206    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5e20e860-569b-4a59-bd0d-c4bac25f9bd3-istio-data\") pod \"e2e-mt-452bd22f-openshift-default-6c77575-g68ls\" (UID: \"5e20e860-569b-4a59-bd0d-c4bac25f9bd3\") " pod="openshift-ingress/e2e-mt-452bd22f-openshift-default-6c77575-g68ls"
Jul 06 18:46:32.218327 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:32.218249    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5e20e860-569b-4a59-bd0d-c4bac25f9bd3-workload-certs\") pod \"e2e-mt-452bd22f-openshift-default-6c77575-g68ls\" (UID: \"5e20e860-569b-4a59-bd0d-c4bac25f9bd3\") " pod="openshift-ingress/e2e-mt-452bd22f-openshift-default-6c77575-g68ls"
Jul 06 18:46:32.218733 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:32.218711    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5e20e860-569b-4a59-bd0d-c4bac25f9bd3-istiod-ca-cert\") pod \"e2e-mt-452bd22f-openshift-default-6c77575-g68ls\" (UID: \"5e20e860-569b-4a59-bd0d-c4bac25f9bd3\") " pod="openshift-ingress/e2e-mt-452bd22f-openshift-default-6c77575-g68ls"
Jul 06 18:46:32.220052 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:32.220003    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5e20e860-569b-4a59-bd0d-c4bac25f9bd3-istio-envoy\") pod \"e2e-mt-452bd22f-openshift-default-6c77575-g68ls\" (UID: \"5e20e860-569b-4a59-bd0d-c4bac25f9bd3\") " pod="openshift-ingress/e2e-mt-452bd22f-openshift-default-6c77575-g68ls"
Jul 06 18:46:32.220199 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:32.220181    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5e20e860-569b-4a59-bd0d-c4bac25f9bd3-istio-podinfo\") pod \"e2e-mt-452bd22f-openshift-default-6c77575-g68ls\" (UID: \"5e20e860-569b-4a59-bd0d-c4bac25f9bd3\") " pod="openshift-ingress/e2e-mt-452bd22f-openshift-default-6c77575-g68ls"
Jul 06 18:46:32.226234 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:32.226197    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5e20e860-569b-4a59-bd0d-c4bac25f9bd3-istio-token\") pod \"e2e-mt-452bd22f-openshift-default-6c77575-g68ls\" (UID: \"5e20e860-569b-4a59-bd0d-c4bac25f9bd3\") " pod="openshift-ingress/e2e-mt-452bd22f-openshift-default-6c77575-g68ls"
Jul 06 18:46:32.226505 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:32.226483    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4ffft\" (UniqueName: \"kubernetes.io/projected/5e20e860-569b-4a59-bd0d-c4bac25f9bd3-kube-api-access-4ffft\") pod \"e2e-mt-452bd22f-openshift-default-6c77575-g68ls\" (UID: \"5e20e860-569b-4a59-bd0d-c4bac25f9bd3\") " pod="openshift-ingress/e2e-mt-452bd22f-openshift-default-6c77575-g68ls"
Jul 06 18:46:32.316939 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:32.316828    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-452bd22f-openshift-default-6c77575-g68ls"
Jul 06 18:46:32.456034 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:32.455993    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-452bd22f-openshift-default-6c77575-g68ls"]
Jul 06 18:46:32.458510 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:46:32.458486    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e20e860_569b_4a59_bd0d_c4bac25f9bd3.slice/crio-086e431ed979ed268a8ab83ffe819af190316c6a28d95d6b37b8bb8439c05123 WatchSource:0}: Error finding container 086e431ed979ed268a8ab83ffe819af190316c6a28d95d6b37b8bb8439c05123: Status 404 returned error can't find the container with id 086e431ed979ed268a8ab83ffe819af190316c6a28d95d6b37b8bb8439c05123
Jul 06 18:46:32.460600 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:32.460564    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 06 18:46:32.460671 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:32.460649    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 06 18:46:32.460713 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:32.460696    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 06 18:46:33.089510 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:33.089465    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-452bd22f-openshift-default-6c77575-g68ls" event={"ID":"5e20e860-569b-4a59-bd0d-c4bac25f9bd3","Type":"ContainerStarted","Data":"fbca3e56f8c4066789227ff36e1970ef3a506eb92ada30d4dde08e4316be8537"}
Jul 06 18:46:33.089987 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:33.089567    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-452bd22f-openshift-default-6c77575-g68ls" event={"ID":"5e20e860-569b-4a59-bd0d-c4bac25f9bd3","Type":"ContainerStarted","Data":"086e431ed979ed268a8ab83ffe819af190316c6a28d95d6b37b8bb8439c05123"}
Jul 06 18:46:33.115237 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:33.115184    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-452bd22f-openshift-default-6c77575-g68ls" podStartSLOduration=2.115169405 podStartE2EDuration="2.115169405s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:46:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:46:33.112194383 +0000 UTC m=+1728.269592868" watchObservedRunningTime="2026-07-06 18:46:33.115169405 +0000 UTC m=+1728.272567890"
Jul 06 18:46:33.323932 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:33.323898    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-452bd22f-openshift-default-6c77575-g68ls"
Jul 06 18:46:33.324094 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:33.323995    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-452bd22f-openshift-default-6c77575-g68ls"
Jul 06 18:46:34.094512 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:34.094479    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-452bd22f-openshift-default-6c77575-g68ls"
Jul 06 18:46:34.095411 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:34.095388    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-452bd22f-openshift-default-6c77575-g68ls"
Jul 06 18:46:35.066957 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:35.066925    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-mt-452bd22f-85d5cb6967-29dks"]
Jul 06 18:46:35.067485 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:35.067465    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="a19fc6cf-31cd-47e5-81bc-77ef9140cc7f" containerName="istio-proxy"
Jul 06 18:46:35.071280 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:35.071255    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-452bd22f-85d5cb6967-29dks"
Jul 06 18:46:35.074344 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:35.074322    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-mt-452bd22f-serving-cert\""
Jul 06 18:46:35.085646 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:35.085620    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-452bd22f-85d5cb6967-29dks"]
Jul 06 18:46:35.143930 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:35.143899    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xmdq7\" (UniqueName: \"kubernetes.io/projected/ab5c8d65-66ad-462f-b284-019b0dd7d25b-kube-api-access-xmdq7\") pod \"maas-api-e2e-mt-452bd22f-85d5cb6967-29dks\" (UID: \"ab5c8d65-66ad-462f-b284-019b0dd7d25b\") " pod="opendatahub/maas-api-e2e-mt-452bd22f-85d5cb6967-29dks"
Jul 06 18:46:35.144326 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:35.144055    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ab5c8d65-66ad-462f-b284-019b0dd7d25b-maas-api-tls\") pod \"maas-api-e2e-mt-452bd22f-85d5cb6967-29dks\" (UID: \"ab5c8d65-66ad-462f-b284-019b0dd7d25b\") " pod="opendatahub/maas-api-e2e-mt-452bd22f-85d5cb6967-29dks"
Jul 06 18:46:35.144326 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:35.144095    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ab5c8d65-66ad-462f-b284-019b0dd7d25b-openshift-service-ca\") pod \"maas-api-e2e-mt-452bd22f-85d5cb6967-29dks\" (UID: \"ab5c8d65-66ad-462f-b284-019b0dd7d25b\") " pod="opendatahub/maas-api-e2e-mt-452bd22f-85d5cb6967-29dks"
Jul 06 18:46:35.244898 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:35.244859    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ab5c8d65-66ad-462f-b284-019b0dd7d25b-maas-api-tls\") pod \"maas-api-e2e-mt-452bd22f-85d5cb6967-29dks\" (UID: \"ab5c8d65-66ad-462f-b284-019b0dd7d25b\") " pod="opendatahub/maas-api-e2e-mt-452bd22f-85d5cb6967-29dks"
Jul 06 18:46:35.245125 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:35.244928    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ab5c8d65-66ad-462f-b284-019b0dd7d25b-openshift-service-ca\") pod \"maas-api-e2e-mt-452bd22f-85d5cb6967-29dks\" (UID: \"ab5c8d65-66ad-462f-b284-019b0dd7d25b\") " pod="opendatahub/maas-api-e2e-mt-452bd22f-85d5cb6967-29dks"
Jul 06 18:46:35.245125 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:35.244989    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xmdq7\" (UniqueName: \"kubernetes.io/projected/ab5c8d65-66ad-462f-b284-019b0dd7d25b-kube-api-access-xmdq7\") pod \"maas-api-e2e-mt-452bd22f-85d5cb6967-29dks\" (UID: \"ab5c8d65-66ad-462f-b284-019b0dd7d25b\") " pod="opendatahub/maas-api-e2e-mt-452bd22f-85d5cb6967-29dks"
Jul 06 18:46:35.245728 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:35.245700    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ab5c8d65-66ad-462f-b284-019b0dd7d25b-openshift-service-ca\") pod \"maas-api-e2e-mt-452bd22f-85d5cb6967-29dks\" (UID: \"ab5c8d65-66ad-462f-b284-019b0dd7d25b\") " pod="opendatahub/maas-api-e2e-mt-452bd22f-85d5cb6967-29dks"
Jul 06 18:46:35.247683 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:35.247658    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ab5c8d65-66ad-462f-b284-019b0dd7d25b-maas-api-tls\") pod \"maas-api-e2e-mt-452bd22f-85d5cb6967-29dks\" (UID: \"ab5c8d65-66ad-462f-b284-019b0dd7d25b\") " pod="opendatahub/maas-api-e2e-mt-452bd22f-85d5cb6967-29dks"
Jul 06 18:46:35.254918 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:35.254889    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xmdq7\" (UniqueName: \"kubernetes.io/projected/ab5c8d65-66ad-462f-b284-019b0dd7d25b-kube-api-access-xmdq7\") pod \"maas-api-e2e-mt-452bd22f-85d5cb6967-29dks\" (UID: \"ab5c8d65-66ad-462f-b284-019b0dd7d25b\") " pod="opendatahub/maas-api-e2e-mt-452bd22f-85d5cb6967-29dks"
Jul 06 18:46:35.382148 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:35.382111    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-452bd22f-85d5cb6967-29dks"
Jul 06 18:46:35.520534 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:35.520506    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-452bd22f-85d5cb6967-29dks"]
Jul 06 18:46:35.522302 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:46:35.522276    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab5c8d65_66ad_462f_b284_019b0dd7d25b.slice/crio-99a32648d1cafe6b2aa1586707afb4fbe1681797130e3488c0c374571e2afc75 WatchSource:0}: Error finding container 99a32648d1cafe6b2aa1586707afb4fbe1681797130e3488c0c374571e2afc75: Status 404 returned error can't find the container with id 99a32648d1cafe6b2aa1586707afb4fbe1681797130e3488c0c374571e2afc75
Jul 06 18:46:36.107201 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:36.107164    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-452bd22f-85d5cb6967-29dks" event={"ID":"ab5c8d65-66ad-462f-b284-019b0dd7d25b","Type":"ContainerStarted","Data":"0945e80e60c93c6699019362eb38bca3edea83d2a43317eaa06667425db66e18"}
Jul 06 18:46:36.107383 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:36.107215    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-452bd22f-85d5cb6967-29dks" event={"ID":"ab5c8d65-66ad-462f-b284-019b0dd7d25b","Type":"ContainerStarted","Data":"99a32648d1cafe6b2aa1586707afb4fbe1681797130e3488c0c374571e2afc75"}
Jul 06 18:46:36.128813 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:36.128740    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-mt-452bd22f-85d5cb6967-29dks" podStartSLOduration=0.751679216 podStartE2EDuration="1.128719743s" totalImagesPullingTime="377.040527ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:46:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:46:36.125874641 +0000 UTC m=+1731.283273127" watchObservedRunningTime="2026-07-06 18:46:36.128719743 +0000 UTC m=+1731.286118228"
Jul 06 18:46:37.111722 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:37.111685    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-mt-452bd22f-85d5cb6967-29dks"
Jul 06 18:46:43.122528 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:43.122497    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-mt-452bd22f-85d5cb6967-29dks"
Jul 06 18:46:46.889888 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:46.889848    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-452bd22f/e2e-lifecycle-model-452bd22f-kserve-6d697f89b7-5qhd7"]
Jul 06 18:46:46.893937 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:46.893914    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-452bd22f/e2e-lifecycle-model-452bd22f-kserve-6d697f89b7-5qhd7"
Jul 06 18:46:46.897666 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:46.897648    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-452bd22f\"/\"kube-root-ca.crt\""
Jul 06 18:46:46.897804 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:46.897650    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-452bd22f\"/\"default-dockercfg-84bn9\""
Jul 06 18:46:46.898882 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:46.898850    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-452bd22f\"/\"e2e-lifecycle-model-452bd22f-kserve-self-signed-certs\""
Jul 06 18:46:46.899005 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:46.898887    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-452bd22f\"/\"openshift-service-ca.crt\""
Jul 06 18:46:46.902809 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:46.902617    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-452bd22f/e2e-lifecycle-model-452bd22f-kserve-6d697f89b7-5qhd7"]
Jul 06 18:46:47.062488 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:47.062440    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/202958d5-b3bd-4089-b704-920f34eabbdc-tmp-dir\") pod \"e2e-lifecycle-model-452bd22f-kserve-6d697f89b7-5qhd7\" (UID: \"202958d5-b3bd-4089-b704-920f34eabbdc\") " pod="ai-tenant-e2e-mt-452bd22f/e2e-lifecycle-model-452bd22f-kserve-6d697f89b7-5qhd7"
Jul 06 18:46:47.062721 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:47.062524    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/202958d5-b3bd-4089-b704-920f34eabbdc-dshm\") pod \"e2e-lifecycle-model-452bd22f-kserve-6d697f89b7-5qhd7\" (UID: \"202958d5-b3bd-4089-b704-920f34eabbdc\") " pod="ai-tenant-e2e-mt-452bd22f/e2e-lifecycle-model-452bd22f-kserve-6d697f89b7-5qhd7"
Jul 06 18:46:47.062721 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:47.062570    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/202958d5-b3bd-4089-b704-920f34eabbdc-model-cache\") pod \"e2e-lifecycle-model-452bd22f-kserve-6d697f89b7-5qhd7\" (UID: \"202958d5-b3bd-4089-b704-920f34eabbdc\") " pod="ai-tenant-e2e-mt-452bd22f/e2e-lifecycle-model-452bd22f-kserve-6d697f89b7-5qhd7"
Jul 06 18:46:47.062721 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:47.062683    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kjz7f\" (UniqueName: \"kubernetes.io/projected/202958d5-b3bd-4089-b704-920f34eabbdc-kube-api-access-kjz7f\") pod \"e2e-lifecycle-model-452bd22f-kserve-6d697f89b7-5qhd7\" (UID: \"202958d5-b3bd-4089-b704-920f34eabbdc\") " pod="ai-tenant-e2e-mt-452bd22f/e2e-lifecycle-model-452bd22f-kserve-6d697f89b7-5qhd7"
Jul 06 18:46:47.062920 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:47.062746    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/202958d5-b3bd-4089-b704-920f34eabbdc-home\") pod \"e2e-lifecycle-model-452bd22f-kserve-6d697f89b7-5qhd7\" (UID: \"202958d5-b3bd-4089-b704-920f34eabbdc\") " pod="ai-tenant-e2e-mt-452bd22f/e2e-lifecycle-model-452bd22f-kserve-6d697f89b7-5qhd7"
Jul 06 18:46:47.062920 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:47.062795    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/202958d5-b3bd-4089-b704-920f34eabbdc-tls-certs\") pod \"e2e-lifecycle-model-452bd22f-kserve-6d697f89b7-5qhd7\" (UID: \"202958d5-b3bd-4089-b704-920f34eabbdc\") " pod="ai-tenant-e2e-mt-452bd22f/e2e-lifecycle-model-452bd22f-kserve-6d697f89b7-5qhd7"
Jul 06 18:46:47.163527 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:47.163437    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kjz7f\" (UniqueName: \"kubernetes.io/projected/202958d5-b3bd-4089-b704-920f34eabbdc-kube-api-access-kjz7f\") pod \"e2e-lifecycle-model-452bd22f-kserve-6d697f89b7-5qhd7\" (UID: \"202958d5-b3bd-4089-b704-920f34eabbdc\") " pod="ai-tenant-e2e-mt-452bd22f/e2e-lifecycle-model-452bd22f-kserve-6d697f89b7-5qhd7"
Jul 06 18:46:47.163527 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:47.163521    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/202958d5-b3bd-4089-b704-920f34eabbdc-home\") pod \"e2e-lifecycle-model-452bd22f-kserve-6d697f89b7-5qhd7\" (UID: \"202958d5-b3bd-4089-b704-920f34eabbdc\") " pod="ai-tenant-e2e-mt-452bd22f/e2e-lifecycle-model-452bd22f-kserve-6d697f89b7-5qhd7"
Jul 06 18:46:47.163743 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:47.163572    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/202958d5-b3bd-4089-b704-920f34eabbdc-tls-certs\") pod \"e2e-lifecycle-model-452bd22f-kserve-6d697f89b7-5qhd7\" (UID: \"202958d5-b3bd-4089-b704-920f34eabbdc\") " pod="ai-tenant-e2e-mt-452bd22f/e2e-lifecycle-model-452bd22f-kserve-6d697f89b7-5qhd7"
Jul 06 18:46:47.163743 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:47.163658    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/202958d5-b3bd-4089-b704-920f34eabbdc-tmp-dir\") pod \"e2e-lifecycle-model-452bd22f-kserve-6d697f89b7-5qhd7\" (UID: \"202958d5-b3bd-4089-b704-920f34eabbdc\") " pod="ai-tenant-e2e-mt-452bd22f/e2e-lifecycle-model-452bd22f-kserve-6d697f89b7-5qhd7"
Jul 06 18:46:47.163743 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:47.163720    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/202958d5-b3bd-4089-b704-920f34eabbdc-dshm\") pod \"e2e-lifecycle-model-452bd22f-kserve-6d697f89b7-5qhd7\" (UID: \"202958d5-b3bd-4089-b704-920f34eabbdc\") " pod="ai-tenant-e2e-mt-452bd22f/e2e-lifecycle-model-452bd22f-kserve-6d697f89b7-5qhd7"
Jul 06 18:46:47.163902 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:47.163747    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/202958d5-b3bd-4089-b704-920f34eabbdc-model-cache\") pod \"e2e-lifecycle-model-452bd22f-kserve-6d697f89b7-5qhd7\" (UID: \"202958d5-b3bd-4089-b704-920f34eabbdc\") " pod="ai-tenant-e2e-mt-452bd22f/e2e-lifecycle-model-452bd22f-kserve-6d697f89b7-5qhd7"
Jul 06 18:46:47.164025 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:47.163977    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/202958d5-b3bd-4089-b704-920f34eabbdc-home\") pod \"e2e-lifecycle-model-452bd22f-kserve-6d697f89b7-5qhd7\" (UID: \"202958d5-b3bd-4089-b704-920f34eabbdc\") " pod="ai-tenant-e2e-mt-452bd22f/e2e-lifecycle-model-452bd22f-kserve-6d697f89b7-5qhd7"
Jul 06 18:46:47.164025 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:47.163996    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/202958d5-b3bd-4089-b704-920f34eabbdc-tmp-dir\") pod \"e2e-lifecycle-model-452bd22f-kserve-6d697f89b7-5qhd7\" (UID: \"202958d5-b3bd-4089-b704-920f34eabbdc\") " pod="ai-tenant-e2e-mt-452bd22f/e2e-lifecycle-model-452bd22f-kserve-6d697f89b7-5qhd7"
Jul 06 18:46:47.164184 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:47.164116    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/202958d5-b3bd-4089-b704-920f34eabbdc-model-cache\") pod \"e2e-lifecycle-model-452bd22f-kserve-6d697f89b7-5qhd7\" (UID: \"202958d5-b3bd-4089-b704-920f34eabbdc\") " pod="ai-tenant-e2e-mt-452bd22f/e2e-lifecycle-model-452bd22f-kserve-6d697f89b7-5qhd7"
Jul 06 18:46:47.165816 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:47.165795    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/202958d5-b3bd-4089-b704-920f34eabbdc-dshm\") pod \"e2e-lifecycle-model-452bd22f-kserve-6d697f89b7-5qhd7\" (UID: \"202958d5-b3bd-4089-b704-920f34eabbdc\") " pod="ai-tenant-e2e-mt-452bd22f/e2e-lifecycle-model-452bd22f-kserve-6d697f89b7-5qhd7"
Jul 06 18:46:47.166124 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:47.166109    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/202958d5-b3bd-4089-b704-920f34eabbdc-tls-certs\") pod \"e2e-lifecycle-model-452bd22f-kserve-6d697f89b7-5qhd7\" (UID: \"202958d5-b3bd-4089-b704-920f34eabbdc\") " pod="ai-tenant-e2e-mt-452bd22f/e2e-lifecycle-model-452bd22f-kserve-6d697f89b7-5qhd7"
Jul 06 18:46:47.173595 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:47.173574    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kjz7f\" (UniqueName: \"kubernetes.io/projected/202958d5-b3bd-4089-b704-920f34eabbdc-kube-api-access-kjz7f\") pod \"e2e-lifecycle-model-452bd22f-kserve-6d697f89b7-5qhd7\" (UID: \"202958d5-b3bd-4089-b704-920f34eabbdc\") " pod="ai-tenant-e2e-mt-452bd22f/e2e-lifecycle-model-452bd22f-kserve-6d697f89b7-5qhd7"
Jul 06 18:46:47.206132 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:47.206097    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-452bd22f/e2e-lifecycle-model-452bd22f-kserve-6d697f89b7-5qhd7"
Jul 06 18:46:47.341682 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:47.341659    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-452bd22f/e2e-lifecycle-model-452bd22f-kserve-6d697f89b7-5qhd7"]
Jul 06 18:46:47.344404 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:46:47.344375    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod202958d5_b3bd_4089_b704_920f34eabbdc.slice/crio-0124d0d7e1377b7127e8e30c2d1d4bc9b6d2ba2ea10e2324fee1d2ff51f2dc67 WatchSource:0}: Error finding container 0124d0d7e1377b7127e8e30c2d1d4bc9b6d2ba2ea10e2324fee1d2ff51f2dc67: Status 404 returned error can't find the container with id 0124d0d7e1377b7127e8e30c2d1d4bc9b6d2ba2ea10e2324fee1d2ff51f2dc67
Jul 06 18:46:48.189890 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:48.189851    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-452bd22f/e2e-lifecycle-model-452bd22f-kserve-6d697f89b7-5qhd7" event={"ID":"202958d5-b3bd-4089-b704-920f34eabbdc","Type":"ContainerStarted","Data":"5d34fc87f42a405d853e59e14f1d98bf900884e18ca37c60fd091fc80a9ed72d"}
Jul 06 18:46:48.190339 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:48.189898    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-452bd22f/e2e-lifecycle-model-452bd22f-kserve-6d697f89b7-5qhd7" event={"ID":"202958d5-b3bd-4089-b704-920f34eabbdc","Type":"ContainerStarted","Data":"0124d0d7e1377b7127e8e30c2d1d4bc9b6d2ba2ea10e2324fee1d2ff51f2dc67"}
Jul 06 18:46:48.191410 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:48.191387    2580 generic.go:356] "Generic (PLEG): container finished" podID="5e20e860-569b-4a59-bd0d-c4bac25f9bd3" containerID="fbca3e56f8c4066789227ff36e1970ef3a506eb92ada30d4dde08e4316be8537" exitCode=0
Jul 06 18:46:48.191510 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:48.191425    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-452bd22f-openshift-default-6c77575-g68ls" event={"ID":"5e20e860-569b-4a59-bd0d-c4bac25f9bd3","Type":"ContainerDied","Data":"fbca3e56f8c4066789227ff36e1970ef3a506eb92ada30d4dde08e4316be8537"}
Jul 06 18:46:48.191757 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:48.191744    2580 scope.go:117] "RemoveContainer" containerID="fbca3e56f8c4066789227ff36e1970ef3a506eb92ada30d4dde08e4316be8537"
Jul 06 18:46:48.192732 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:48.192705    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 06 18:46:48.192795 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:48.192768    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 06 18:46:48.192834 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:48.192800    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 06 18:46:48.212852 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:48.212797    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-452bd22f/e2e-lifecycle-model-452bd22f-kserve-6d697f89b7-5qhd7" podStartSLOduration=2.212778931 podStartE2EDuration="2.212778931s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:46:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:46:48.209524382 +0000 UTC m=+1743.366922868" watchObservedRunningTime="2026-07-06 18:46:48.212778931 +0000 UTC m=+1743.370177417"
Jul 06 18:46:49.094722 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:49.094689    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-452bd22f-openshift-default-6c77575-g68ls"
Jul 06 18:46:49.199434 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:49.199389    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-452bd22f-openshift-default-6c77575-g68ls" event={"ID":"5e20e860-569b-4a59-bd0d-c4bac25f9bd3","Type":"ContainerStarted","Data":"5077e810369819f7e1661965cf2053680e790b336f092c333fff5c85b350c053"}
Jul 06 18:46:49.317607 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:49.317574    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-452bd22f-openshift-default-6c77575-g68ls"
Jul 06 18:46:49.318727 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:49.318699    2580 patch_prober.go:28] interesting pod/e2e-mt-452bd22f-openshift-default-6c77575-g68ls container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Jul 06 18:46:49.318844 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:49.318760    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-452bd22f-openshift-default-6c77575-g68ls" podUID="5e20e860-569b-4a59-bd0d-c4bac25f9bd3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Jul 06 18:46:50.317741 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:50.317702    2580 patch_prober.go:28] interesting pod/e2e-mt-452bd22f-openshift-default-6c77575-g68ls container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Jul 06 18:46:50.318187 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:50.317761    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-452bd22f-openshift-default-6c77575-g68ls" podUID="5e20e860-569b-4a59-bd0d-c4bac25f9bd3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Jul 06 18:46:51.318010 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:51.317972    2580 patch_prober.go:28] interesting pod/e2e-mt-452bd22f-openshift-default-6c77575-g68ls container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Jul 06 18:46:51.318408 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:51.318063    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-452bd22f-openshift-default-6c77575-g68ls" podUID="5e20e860-569b-4a59-bd0d-c4bac25f9bd3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Jul 06 18:46:52.317584 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:52.317542    2580 patch_prober.go:28] interesting pod/e2e-mt-452bd22f-openshift-default-6c77575-g68ls container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Jul 06 18:46:52.317768 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:52.317615    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-452bd22f-openshift-default-6c77575-g68ls" podUID="5e20e860-569b-4a59-bd0d-c4bac25f9bd3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Jul 06 18:46:53.317530 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:53.317483    2580 patch_prober.go:28] interesting pod/e2e-mt-452bd22f-openshift-default-6c77575-g68ls container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Jul 06 18:46:53.318007 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:53.317941    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-452bd22f-openshift-default-6c77575-g68ls" podUID="5e20e860-569b-4a59-bd0d-c4bac25f9bd3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Jul 06 18:46:54.321332 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:54.321301    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-452bd22f-openshift-default-6c77575-g68ls"
Jul 06 18:46:54.321718 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:54.321561    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-452bd22f-openshift-default-6c77575-g68ls"
Jul 06 18:46:54.322340 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:54.322320    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-452bd22f-openshift-default-6c77575-g68ls"
Jul 06 18:46:57.206562 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:57.206516    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-452bd22f/e2e-lifecycle-model-452bd22f-kserve-6d697f89b7-5qhd7"
Jul 06 18:46:57.207062 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:57.206577    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-452bd22f/e2e-lifecycle-model-452bd22f-kserve-6d697f89b7-5qhd7"
Jul 06 18:46:57.219312 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:57.219289    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-452bd22f/e2e-lifecycle-model-452bd22f-kserve-6d697f89b7-5qhd7"
Jul 06 18:46:57.248242 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:46:57.248217    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-452bd22f/e2e-lifecycle-model-452bd22f-kserve-6d697f89b7-5qhd7"
Jul 06 18:47:12.295778 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:47:12.295739    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-452bd22f-85d5cb6967-29dks"]
Jul 06 18:47:12.296376 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:47:12.296007    2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-mt-452bd22f-85d5cb6967-29dks" podUID="ab5c8d65-66ad-462f-b284-019b0dd7d25b" containerName="maas-api" containerID="cri-o://0945e80e60c93c6699019362eb38bca3edea83d2a43317eaa06667425db66e18" gracePeriod=30
Jul 06 18:47:12.546932 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:47:12.546867    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-452bd22f-85d5cb6967-29dks"
Jul 06 18:47:12.600058 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:47:12.600024    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ab5c8d65-66ad-462f-b284-019b0dd7d25b-maas-api-tls\") pod \"ab5c8d65-66ad-462f-b284-019b0dd7d25b\" (UID: \"ab5c8d65-66ad-462f-b284-019b0dd7d25b\") "
Jul 06 18:47:12.600247 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:47:12.600119    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ab5c8d65-66ad-462f-b284-019b0dd7d25b-openshift-service-ca\") pod \"ab5c8d65-66ad-462f-b284-019b0dd7d25b\" (UID: \"ab5c8d65-66ad-462f-b284-019b0dd7d25b\") "
Jul 06 18:47:12.600247 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:47:12.600193    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xmdq7\" (UniqueName: \"kubernetes.io/projected/ab5c8d65-66ad-462f-b284-019b0dd7d25b-kube-api-access-xmdq7\") pod \"ab5c8d65-66ad-462f-b284-019b0dd7d25b\" (UID: \"ab5c8d65-66ad-462f-b284-019b0dd7d25b\") "
Jul 06 18:47:12.600478 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:47:12.600451    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ab5c8d65-66ad-462f-b284-019b0dd7d25b-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ab5c8d65-66ad-462f-b284-019b0dd7d25b" (UID: "ab5c8d65-66ad-462f-b284-019b0dd7d25b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 18:47:12.601962 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:47:12.601939    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ab5c8d65-66ad-462f-b284-019b0dd7d25b-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ab5c8d65-66ad-462f-b284-019b0dd7d25b" (UID: "ab5c8d65-66ad-462f-b284-019b0dd7d25b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 18:47:12.602174 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:47:12.602157    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ab5c8d65-66ad-462f-b284-019b0dd7d25b-kube-api-access-xmdq7" (OuterVolumeSpecName: "kube-api-access-xmdq7") pod "ab5c8d65-66ad-462f-b284-019b0dd7d25b" (UID: "ab5c8d65-66ad-462f-b284-019b0dd7d25b"). InnerVolumeSpecName "kube-api-access-xmdq7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 18:47:12.701647 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:47:12.701611    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ab5c8d65-66ad-462f-b284-019b0dd7d25b-openshift-service-ca\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:47:12.701647 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:47:12.701640    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xmdq7\" (UniqueName: \"kubernetes.io/projected/ab5c8d65-66ad-462f-b284-019b0dd7d25b-kube-api-access-xmdq7\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:47:12.701647 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:47:12.701649    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ab5c8d65-66ad-462f-b284-019b0dd7d25b-maas-api-tls\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:47:13.314507 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:47:13.314469    2580 generic.go:356] "Generic (PLEG): container finished" podID="ab5c8d65-66ad-462f-b284-019b0dd7d25b" containerID="0945e80e60c93c6699019362eb38bca3edea83d2a43317eaa06667425db66e18" exitCode=0
Jul 06 18:47:13.314507 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:47:13.314508    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-452bd22f-85d5cb6967-29dks" event={"ID":"ab5c8d65-66ad-462f-b284-019b0dd7d25b","Type":"ContainerDied","Data":"0945e80e60c93c6699019362eb38bca3edea83d2a43317eaa06667425db66e18"}
Jul 06 18:47:13.315355 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:47:13.314522    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-452bd22f-85d5cb6967-29dks"
Jul 06 18:47:13.315355 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:47:13.314563    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-452bd22f-85d5cb6967-29dks" event={"ID":"ab5c8d65-66ad-462f-b284-019b0dd7d25b","Type":"ContainerDied","Data":"99a32648d1cafe6b2aa1586707afb4fbe1681797130e3488c0c374571e2afc75"}
Jul 06 18:47:13.315355 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:47:13.314595    2580 scope.go:117] "RemoveContainer" containerID="0945e80e60c93c6699019362eb38bca3edea83d2a43317eaa06667425db66e18"
Jul 06 18:47:13.323666 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:47:13.323600    2580 scope.go:117] "RemoveContainer" containerID="0945e80e60c93c6699019362eb38bca3edea83d2a43317eaa06667425db66e18"
Jul 06 18:47:13.324235 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:47:13.324211    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0945e80e60c93c6699019362eb38bca3edea83d2a43317eaa06667425db66e18\": container with ID starting with 0945e80e60c93c6699019362eb38bca3edea83d2a43317eaa06667425db66e18 not found: ID does not exist" containerID="0945e80e60c93c6699019362eb38bca3edea83d2a43317eaa06667425db66e18"
Jul 06 18:47:13.324340 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:47:13.324243    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0945e80e60c93c6699019362eb38bca3edea83d2a43317eaa06667425db66e18"} err="failed to get container status \"0945e80e60c93c6699019362eb38bca3edea83d2a43317eaa06667425db66e18\": rpc error: code = NotFound desc = could not find container \"0945e80e60c93c6699019362eb38bca3edea83d2a43317eaa06667425db66e18\": container with ID starting with 0945e80e60c93c6699019362eb38bca3edea83d2a43317eaa06667425db66e18 not found: ID does not exist"
Jul 06 18:47:13.340448 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:47:13.340422    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-452bd22f-85d5cb6967-29dks"]
Jul 06 18:47:13.342310 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:47:13.342291    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-mt-452bd22f-85d5cb6967-29dks"]
Jul 06 18:47:15.323107 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:47:15.323068    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ab5c8d65-66ad-462f-b284-019b0dd7d25b" path="/var/lib/kubelet/pods/ab5c8d65-66ad-462f-b284-019b0dd7d25b/volumes"
Jul 06 18:47:23.048455 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:47:23.048416    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-452bd22f/e2e-lifecycle-model-452bd22f-kserve-6d697f89b7-5qhd7"]
Jul 06 18:47:38.070904 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:47:38.070790    2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-mt-452bd22f/e2e-lifecycle-model-452bd22f-kserve-6d697f89b7-5qhd7" podUID="202958d5-b3bd-4089-b704-920f34eabbdc" containerName="main" containerID="cri-o://5d34fc87f42a405d853e59e14f1d98bf900884e18ca37c60fd091fc80a9ed72d" gracePeriod=45
Jul 06 18:47:38.333909 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:47:38.333883    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-452bd22f/e2e-lifecycle-model-452bd22f-kserve-6d697f89b7-5qhd7"
Jul 06 18:47:38.428728 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:47:38.428693    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/202958d5-b3bd-4089-b704-920f34eabbdc-model-cache\") pod \"202958d5-b3bd-4089-b704-920f34eabbdc\" (UID: \"202958d5-b3bd-4089-b704-920f34eabbdc\") "
Jul 06 18:47:38.428928 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:47:38.428740    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/202958d5-b3bd-4089-b704-920f34eabbdc-tmp-dir\") pod \"202958d5-b3bd-4089-b704-920f34eabbdc\" (UID: \"202958d5-b3bd-4089-b704-920f34eabbdc\") "
Jul 06 18:47:38.428928 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:47:38.428773    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/202958d5-b3bd-4089-b704-920f34eabbdc-home\") pod \"202958d5-b3bd-4089-b704-920f34eabbdc\" (UID: \"202958d5-b3bd-4089-b704-920f34eabbdc\") "
Jul 06 18:47:38.428928 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:47:38.428800    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/202958d5-b3bd-4089-b704-920f34eabbdc-dshm\") pod \"202958d5-b3bd-4089-b704-920f34eabbdc\" (UID: \"202958d5-b3bd-4089-b704-920f34eabbdc\") "
Jul 06 18:47:38.428928 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:47:38.428835    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kjz7f\" (UniqueName: \"kubernetes.io/projected/202958d5-b3bd-4089-b704-920f34eabbdc-kube-api-access-kjz7f\") pod \"202958d5-b3bd-4089-b704-920f34eabbdc\" (UID: \"202958d5-b3bd-4089-b704-920f34eabbdc\") "
Jul 06 18:47:38.428928 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:47:38.428867    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/202958d5-b3bd-4089-b704-920f34eabbdc-tls-certs\") pod \"202958d5-b3bd-4089-b704-920f34eabbdc\" (UID: \"202958d5-b3bd-4089-b704-920f34eabbdc\") "
Jul 06 18:47:38.429527 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:47:38.429355    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/202958d5-b3bd-4089-b704-920f34eabbdc-home" (OuterVolumeSpecName: "home") pod "202958d5-b3bd-4089-b704-920f34eabbdc" (UID: "202958d5-b3bd-4089-b704-920f34eabbdc"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 18:47:38.429527 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:47:38.429460    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/202958d5-b3bd-4089-b704-920f34eabbdc-model-cache" (OuterVolumeSpecName: "model-cache") pod "202958d5-b3bd-4089-b704-920f34eabbdc" (UID: "202958d5-b3bd-4089-b704-920f34eabbdc"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 18:47:38.429527 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:47:38.429479    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/202958d5-b3bd-4089-b704-920f34eabbdc-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "202958d5-b3bd-4089-b704-920f34eabbdc" (UID: "202958d5-b3bd-4089-b704-920f34eabbdc"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 18:47:38.429773 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:47:38.429645    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/202958d5-b3bd-4089-b704-920f34eabbdc-model-cache\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:47:38.429773 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:47:38.429665    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/202958d5-b3bd-4089-b704-920f34eabbdc-tmp-dir\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:47:38.429773 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:47:38.429678    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/202958d5-b3bd-4089-b704-920f34eabbdc-home\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:47:38.432037 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:47:38.431988    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/202958d5-b3bd-4089-b704-920f34eabbdc-kube-api-access-kjz7f" (OuterVolumeSpecName: "kube-api-access-kjz7f") pod "202958d5-b3bd-4089-b704-920f34eabbdc" (UID: "202958d5-b3bd-4089-b704-920f34eabbdc"). InnerVolumeSpecName "kube-api-access-kjz7f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 18:47:38.432152 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:47:38.432076    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/202958d5-b3bd-4089-b704-920f34eabbdc-dshm" (OuterVolumeSpecName: "dshm") pod "202958d5-b3bd-4089-b704-920f34eabbdc" (UID: "202958d5-b3bd-4089-b704-920f34eabbdc"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 18:47:38.434075 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:47:38.432431    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/202958d5-b3bd-4089-b704-920f34eabbdc-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "202958d5-b3bd-4089-b704-920f34eabbdc" (UID: "202958d5-b3bd-4089-b704-920f34eabbdc"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 18:47:38.442764 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:47:38.442732    2580 generic.go:356] "Generic (PLEG): container finished" podID="202958d5-b3bd-4089-b704-920f34eabbdc" containerID="5d34fc87f42a405d853e59e14f1d98bf900884e18ca37c60fd091fc80a9ed72d" exitCode=0
Jul 06 18:47:38.442853 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:47:38.442782    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-452bd22f/e2e-lifecycle-model-452bd22f-kserve-6d697f89b7-5qhd7"
Jul 06 18:47:38.442853 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:47:38.442804    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-452bd22f/e2e-lifecycle-model-452bd22f-kserve-6d697f89b7-5qhd7" event={"ID":"202958d5-b3bd-4089-b704-920f34eabbdc","Type":"ContainerDied","Data":"5d34fc87f42a405d853e59e14f1d98bf900884e18ca37c60fd091fc80a9ed72d"}
Jul 06 18:47:38.442853 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:47:38.442837    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-452bd22f/e2e-lifecycle-model-452bd22f-kserve-6d697f89b7-5qhd7" event={"ID":"202958d5-b3bd-4089-b704-920f34eabbdc","Type":"ContainerDied","Data":"0124d0d7e1377b7127e8e30c2d1d4bc9b6d2ba2ea10e2324fee1d2ff51f2dc67"}
Jul 06 18:47:38.442959 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:47:38.442855    2580 scope.go:117] "RemoveContainer" containerID="5d34fc87f42a405d853e59e14f1d98bf900884e18ca37c60fd091fc80a9ed72d"
Jul 06 18:47:38.451666 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:47:38.451651    2580 scope.go:117] "RemoveContainer" containerID="5d34fc87f42a405d853e59e14f1d98bf900884e18ca37c60fd091fc80a9ed72d"
Jul 06 18:47:38.451919 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:47:38.451899    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5d34fc87f42a405d853e59e14f1d98bf900884e18ca37c60fd091fc80a9ed72d\": container with ID starting with 5d34fc87f42a405d853e59e14f1d98bf900884e18ca37c60fd091fc80a9ed72d not found: ID does not exist" containerID="5d34fc87f42a405d853e59e14f1d98bf900884e18ca37c60fd091fc80a9ed72d"
Jul 06 18:47:38.451984 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:47:38.451932    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5d34fc87f42a405d853e59e14f1d98bf900884e18ca37c60fd091fc80a9ed72d"} err="failed to get container status \"5d34fc87f42a405d853e59e14f1d98bf900884e18ca37c60fd091fc80a9ed72d\": rpc error: code = NotFound desc = could not find container \"5d34fc87f42a405d853e59e14f1d98bf900884e18ca37c60fd091fc80a9ed72d\": container with ID starting with 5d34fc87f42a405d853e59e14f1d98bf900884e18ca37c60fd091fc80a9ed72d not found: ID does not exist"
Jul 06 18:47:38.464198 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:47:38.464175    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-452bd22f/e2e-lifecycle-model-452bd22f-kserve-6d697f89b7-5qhd7"]
Jul 06 18:47:38.467721 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:47:38.467699    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-452bd22f/e2e-lifecycle-model-452bd22f-kserve-6d697f89b7-5qhd7"]
Jul 06 18:47:38.530930 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:47:38.530901    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/202958d5-b3bd-4089-b704-920f34eabbdc-dshm\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:47:38.530930 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:47:38.530925    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kjz7f\" (UniqueName: \"kubernetes.io/projected/202958d5-b3bd-4089-b704-920f34eabbdc-kube-api-access-kjz7f\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:47:38.531149 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:47:38.530936    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/202958d5-b3bd-4089-b704-920f34eabbdc-tls-certs\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:47:39.324034 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:47:39.323994    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="202958d5-b3bd-4089-b704-920f34eabbdc" path="/var/lib/kubelet/pods/202958d5-b3bd-4089-b704-920f34eabbdc/volumes"
Jul 06 18:48:12.643314 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:48:12.643277    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-452bd22f-openshift-default-6c77575-g68ls"]
Jul 06 18:48:12.643770 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:48:12.643527    2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-452bd22f-openshift-default-6c77575-g68ls" podUID="5e20e860-569b-4a59-bd0d-c4bac25f9bd3" containerName="istio-proxy" containerID="cri-o://5077e810369819f7e1661965cf2053680e790b336f092c333fff5c85b350c053" gracePeriod=30
Jul 06 18:48:17.902427 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:48:17.902401    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-452bd22f-openshift-default-6c77575-g68ls"
Jul 06 18:48:17.974720 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:48:17.974620    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e20e860-569b-4a59-bd0d-c4bac25f9bd3-credential-socket\") pod \"5e20e860-569b-4a59-bd0d-c4bac25f9bd3\" (UID: \"5e20e860-569b-4a59-bd0d-c4bac25f9bd3\") "
Jul 06 18:48:17.974720 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:48:17.974686    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5e20e860-569b-4a59-bd0d-c4bac25f9bd3-istio-envoy\") pod \"5e20e860-569b-4a59-bd0d-c4bac25f9bd3\" (UID: \"5e20e860-569b-4a59-bd0d-c4bac25f9bd3\") "
Jul 06 18:48:17.974720 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:48:17.974717    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5e20e860-569b-4a59-bd0d-c4bac25f9bd3-istio-podinfo\") pod \"5e20e860-569b-4a59-bd0d-c4bac25f9bd3\" (UID: \"5e20e860-569b-4a59-bd0d-c4bac25f9bd3\") "
Jul 06 18:48:17.975008 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:48:17.974757    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5e20e860-569b-4a59-bd0d-c4bac25f9bd3-istio-data\") pod \"5e20e860-569b-4a59-bd0d-c4bac25f9bd3\" (UID: \"5e20e860-569b-4a59-bd0d-c4bac25f9bd3\") "
Jul 06 18:48:17.975008 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:48:17.974784    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5e20e860-569b-4a59-bd0d-c4bac25f9bd3-istio-token\") pod \"5e20e860-569b-4a59-bd0d-c4bac25f9bd3\" (UID: \"5e20e860-569b-4a59-bd0d-c4bac25f9bd3\") "
Jul 06 18:48:17.975008 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:48:17.974814    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4ffft\" (UniqueName: \"kubernetes.io/projected/5e20e860-569b-4a59-bd0d-c4bac25f9bd3-kube-api-access-4ffft\") pod \"5e20e860-569b-4a59-bd0d-c4bac25f9bd3\" (UID: \"5e20e860-569b-4a59-bd0d-c4bac25f9bd3\") "
Jul 06 18:48:17.975008 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:48:17.974882    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e20e860-569b-4a59-bd0d-c4bac25f9bd3-workload-socket\") pod \"5e20e860-569b-4a59-bd0d-c4bac25f9bd3\" (UID: \"5e20e860-569b-4a59-bd0d-c4bac25f9bd3\") "
Jul 06 18:48:17.975008 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:48:17.974918    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e20e860-569b-4a59-bd0d-c4bac25f9bd3-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "5e20e860-569b-4a59-bd0d-c4bac25f9bd3" (UID: "5e20e860-569b-4a59-bd0d-c4bac25f9bd3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 18:48:17.975296 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:48:17.975133    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5e20e860-569b-4a59-bd0d-c4bac25f9bd3-istiod-ca-cert\") pod \"5e20e860-569b-4a59-bd0d-c4bac25f9bd3\" (UID: \"5e20e860-569b-4a59-bd0d-c4bac25f9bd3\") "
Jul 06 18:48:17.975296 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:48:17.975204    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5e20e860-569b-4a59-bd0d-c4bac25f9bd3-workload-certs\") pod \"5e20e860-569b-4a59-bd0d-c4bac25f9bd3\" (UID: \"5e20e860-569b-4a59-bd0d-c4bac25f9bd3\") "
Jul 06 18:48:17.975607 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:48:17.975559    2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e20e860-569b-4a59-bd0d-c4bac25f9bd3-credential-socket\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:48:17.976289 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:48:17.976242    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e20e860-569b-4a59-bd0d-c4bac25f9bd3-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "5e20e860-569b-4a59-bd0d-c4bac25f9bd3" (UID: "5e20e860-569b-4a59-bd0d-c4bac25f9bd3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 18:48:17.976397 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:48:17.976282    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5e20e860-569b-4a59-bd0d-c4bac25f9bd3-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "5e20e860-569b-4a59-bd0d-c4bac25f9bd3" (UID: "5e20e860-569b-4a59-bd0d-c4bac25f9bd3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 18:48:17.976397 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:48:17.976349    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e20e860-569b-4a59-bd0d-c4bac25f9bd3-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "5e20e860-569b-4a59-bd0d-c4bac25f9bd3" (UID: "5e20e860-569b-4a59-bd0d-c4bac25f9bd3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 18:48:17.976397 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:48:17.976368    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e20e860-569b-4a59-bd0d-c4bac25f9bd3-istio-data" (OuterVolumeSpecName: "istio-data") pod "5e20e860-569b-4a59-bd0d-c4bac25f9bd3" (UID: "5e20e860-569b-4a59-bd0d-c4bac25f9bd3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 18:48:17.977503 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:48:17.977468    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e20e860-569b-4a59-bd0d-c4bac25f9bd3-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "5e20e860-569b-4a59-bd0d-c4bac25f9bd3" (UID: "5e20e860-569b-4a59-bd0d-c4bac25f9bd3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 18:48:17.977624 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:48:17.977528    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5e20e860-569b-4a59-bd0d-c4bac25f9bd3-istio-token" (OuterVolumeSpecName: "istio-token") pod "5e20e860-569b-4a59-bd0d-c4bac25f9bd3" (UID: "5e20e860-569b-4a59-bd0d-c4bac25f9bd3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 18:48:17.977690 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:48:17.977672    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5e20e860-569b-4a59-bd0d-c4bac25f9bd3-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "5e20e860-569b-4a59-bd0d-c4bac25f9bd3" (UID: "5e20e860-569b-4a59-bd0d-c4bac25f9bd3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 06 18:48:17.978612 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:48:17.978591    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5e20e860-569b-4a59-bd0d-c4bac25f9bd3-kube-api-access-4ffft" (OuterVolumeSpecName: "kube-api-access-4ffft") pod "5e20e860-569b-4a59-bd0d-c4bac25f9bd3" (UID: "5e20e860-569b-4a59-bd0d-c4bac25f9bd3"). InnerVolumeSpecName "kube-api-access-4ffft". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 18:48:18.076563 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:48:18.076519    2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5e20e860-569b-4a59-bd0d-c4bac25f9bd3-workload-certs\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:48:18.076563 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:48:18.076557    2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5e20e860-569b-4a59-bd0d-c4bac25f9bd3-istio-envoy\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:48:18.076563 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:48:18.076569    2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5e20e860-569b-4a59-bd0d-c4bac25f9bd3-istio-podinfo\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:48:18.076563 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:48:18.076578    2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5e20e860-569b-4a59-bd0d-c4bac25f9bd3-istio-data\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:48:18.076846 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:48:18.076588    2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5e20e860-569b-4a59-bd0d-c4bac25f9bd3-istio-token\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:48:18.076846 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:48:18.076596    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4ffft\" (UniqueName: \"kubernetes.io/projected/5e20e860-569b-4a59-bd0d-c4bac25f9bd3-kube-api-access-4ffft\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:48:18.076846 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:48:18.076606    2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e20e860-569b-4a59-bd0d-c4bac25f9bd3-workload-socket\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:48:18.076846 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:48:18.076614    2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5e20e860-569b-4a59-bd0d-c4bac25f9bd3-istiod-ca-cert\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:48:18.634584 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:48:18.634552    2580 generic.go:356] "Generic (PLEG): container finished" podID="5e20e860-569b-4a59-bd0d-c4bac25f9bd3" containerID="5077e810369819f7e1661965cf2053680e790b336f092c333fff5c85b350c053" exitCode=0
Jul 06 18:48:18.634753 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:48:18.634609    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-452bd22f-openshift-default-6c77575-g68ls"
Jul 06 18:48:18.634753 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:48:18.634636    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-452bd22f-openshift-default-6c77575-g68ls" event={"ID":"5e20e860-569b-4a59-bd0d-c4bac25f9bd3","Type":"ContainerDied","Data":"5077e810369819f7e1661965cf2053680e790b336f092c333fff5c85b350c053"}
Jul 06 18:48:18.634753 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:48:18.634701    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-452bd22f-openshift-default-6c77575-g68ls" event={"ID":"5e20e860-569b-4a59-bd0d-c4bac25f9bd3","Type":"ContainerDied","Data":"086e431ed979ed268a8ab83ffe819af190316c6a28d95d6b37b8bb8439c05123"}
Jul 06 18:48:18.634753 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:48:18.634731    2580 scope.go:117] "RemoveContainer" containerID="5077e810369819f7e1661965cf2053680e790b336f092c333fff5c85b350c053"
Jul 06 18:48:18.644328 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:48:18.644304    2580 scope.go:117] "RemoveContainer" containerID="fbca3e56f8c4066789227ff36e1970ef3a506eb92ada30d4dde08e4316be8537"
Jul 06 18:48:18.652158 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:48:18.652139    2580 scope.go:117] "RemoveContainer" containerID="5077e810369819f7e1661965cf2053680e790b336f092c333fff5c85b350c053"
Jul 06 18:48:18.652480 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:48:18.652448    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5077e810369819f7e1661965cf2053680e790b336f092c333fff5c85b350c053\": container with ID starting with 5077e810369819f7e1661965cf2053680e790b336f092c333fff5c85b350c053 not found: ID does not exist" containerID="5077e810369819f7e1661965cf2053680e790b336f092c333fff5c85b350c053"
Jul 06 18:48:18.652612 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:48:18.652493    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5077e810369819f7e1661965cf2053680e790b336f092c333fff5c85b350c053"} err="failed to get container status \"5077e810369819f7e1661965cf2053680e790b336f092c333fff5c85b350c053\": rpc error: code = NotFound desc = could not find container \"5077e810369819f7e1661965cf2053680e790b336f092c333fff5c85b350c053\": container with ID starting with 5077e810369819f7e1661965cf2053680e790b336f092c333fff5c85b350c053 not found: ID does not exist"
Jul 06 18:48:18.652612 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:48:18.652519    2580 scope.go:117] "RemoveContainer" containerID="fbca3e56f8c4066789227ff36e1970ef3a506eb92ada30d4dde08e4316be8537"
Jul 06 18:48:18.652801 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:48:18.652780    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fbca3e56f8c4066789227ff36e1970ef3a506eb92ada30d4dde08e4316be8537\": container with ID starting with fbca3e56f8c4066789227ff36e1970ef3a506eb92ada30d4dde08e4316be8537 not found: ID does not exist" containerID="fbca3e56f8c4066789227ff36e1970ef3a506eb92ada30d4dde08e4316be8537"
Jul 06 18:48:18.652864 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:48:18.652807    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fbca3e56f8c4066789227ff36e1970ef3a506eb92ada30d4dde08e4316be8537"} err="failed to get container status \"fbca3e56f8c4066789227ff36e1970ef3a506eb92ada30d4dde08e4316be8537\": rpc error: code = NotFound desc = could not find container \"fbca3e56f8c4066789227ff36e1970ef3a506eb92ada30d4dde08e4316be8537\": container with ID starting with fbca3e56f8c4066789227ff36e1970ef3a506eb92ada30d4dde08e4316be8537 not found: ID does not exist"
Jul 06 18:48:18.672223 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:48:18.672193    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-452bd22f-openshift-default-6c77575-g68ls"]
Jul 06 18:48:18.675238 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:48:18.675215    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-452bd22f-openshift-default-6c77575-g68ls"]
Jul 06 18:48:19.323254 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:48:19.323214    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5e20e860-569b-4a59-bd0d-c4bac25f9bd3" path="/var/lib/kubelet/pods/5e20e860-569b-4a59-bd0d-c4bac25f9bd3/volumes"
Jul 06 18:49:11.026756 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:11.026667    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-203de8-openshift-default-5f878c54b5-8c99l"]
Jul 06 18:49:11.027264 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:11.027075    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5e20e860-569b-4a59-bd0d-c4bac25f9bd3" containerName="istio-proxy"
Jul 06 18:49:11.027264 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:11.027089    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="5e20e860-569b-4a59-bd0d-c4bac25f9bd3" containerName="istio-proxy"
Jul 06 18:49:11.027264 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:11.027105    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ab5c8d65-66ad-462f-b284-019b0dd7d25b" containerName="maas-api"
Jul 06 18:49:11.027264 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:11.027111    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="ab5c8d65-66ad-462f-b284-019b0dd7d25b" containerName="maas-api"
Jul 06 18:49:11.027264 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:11.027127    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="202958d5-b3bd-4089-b704-920f34eabbdc" containerName="main"
Jul 06 18:49:11.027264 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:11.027132    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="202958d5-b3bd-4089-b704-920f34eabbdc" containerName="main"
Jul 06 18:49:11.027264 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:11.027185    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="5e20e860-569b-4a59-bd0d-c4bac25f9bd3" containerName="istio-proxy"
Jul 06 18:49:11.027264 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:11.027192    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="ab5c8d65-66ad-462f-b284-019b0dd7d25b" containerName="maas-api"
Jul 06 18:49:11.027264 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:11.027201    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="5e20e860-569b-4a59-bd0d-c4bac25f9bd3" containerName="istio-proxy"
Jul 06 18:49:11.027264 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:11.027207    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="202958d5-b3bd-4089-b704-920f34eabbdc" containerName="main"
Jul 06 18:49:11.030240 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:11.030225    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-203de8-openshift-default-5f878c54b5-8c99l"
Jul 06 18:49:11.033470 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:11.033443    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-203de8-openshift-default-dockercfg-g2wmx\""
Jul 06 18:49:11.060910 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:11.060875    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-203de8-openshift-default-5f878c54b5-8c99l"]
Jul 06 18:49:11.136842 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:11.136804    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/38d03a05-5c33-47fe-ad1d-6866511596bf-istio-data\") pod \"e2e-inf-a-203de8-openshift-default-5f878c54b5-8c99l\" (UID: \"38d03a05-5c33-47fe-ad1d-6866511596bf\") " pod="openshift-ingress/e2e-inf-a-203de8-openshift-default-5f878c54b5-8c99l"
Jul 06 18:49:11.136842 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:11.136841    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/38d03a05-5c33-47fe-ad1d-6866511596bf-istiod-ca-cert\") pod \"e2e-inf-a-203de8-openshift-default-5f878c54b5-8c99l\" (UID: \"38d03a05-5c33-47fe-ad1d-6866511596bf\") " pod="openshift-ingress/e2e-inf-a-203de8-openshift-default-5f878c54b5-8c99l"
Jul 06 18:49:11.137097 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:11.136875    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/38d03a05-5c33-47fe-ad1d-6866511596bf-workload-socket\") pod \"e2e-inf-a-203de8-openshift-default-5f878c54b5-8c99l\" (UID: \"38d03a05-5c33-47fe-ad1d-6866511596bf\") " pod="openshift-ingress/e2e-inf-a-203de8-openshift-default-5f878c54b5-8c99l"
Jul 06 18:49:11.137097 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:11.136952    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/38d03a05-5c33-47fe-ad1d-6866511596bf-workload-certs\") pod \"e2e-inf-a-203de8-openshift-default-5f878c54b5-8c99l\" (UID: \"38d03a05-5c33-47fe-ad1d-6866511596bf\") " pod="openshift-ingress/e2e-inf-a-203de8-openshift-default-5f878c54b5-8c99l"
Jul 06 18:49:11.137097 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:11.136984    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pxgnr\" (UniqueName: \"kubernetes.io/projected/38d03a05-5c33-47fe-ad1d-6866511596bf-kube-api-access-pxgnr\") pod \"e2e-inf-a-203de8-openshift-default-5f878c54b5-8c99l\" (UID: \"38d03a05-5c33-47fe-ad1d-6866511596bf\") " pod="openshift-ingress/e2e-inf-a-203de8-openshift-default-5f878c54b5-8c99l"
Jul 06 18:49:11.137097 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:11.137076    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/38d03a05-5c33-47fe-ad1d-6866511596bf-istio-token\") pod \"e2e-inf-a-203de8-openshift-default-5f878c54b5-8c99l\" (UID: \"38d03a05-5c33-47fe-ad1d-6866511596bf\") " pod="openshift-ingress/e2e-inf-a-203de8-openshift-default-5f878c54b5-8c99l"
Jul 06 18:49:11.137244 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:11.137118    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/38d03a05-5c33-47fe-ad1d-6866511596bf-istio-podinfo\") pod \"e2e-inf-a-203de8-openshift-default-5f878c54b5-8c99l\" (UID: \"38d03a05-5c33-47fe-ad1d-6866511596bf\") " pod="openshift-ingress/e2e-inf-a-203de8-openshift-default-5f878c54b5-8c99l"
Jul 06 18:49:11.137244 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:11.137139    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/38d03a05-5c33-47fe-ad1d-6866511596bf-credential-socket\") pod \"e2e-inf-a-203de8-openshift-default-5f878c54b5-8c99l\" (UID: \"38d03a05-5c33-47fe-ad1d-6866511596bf\") " pod="openshift-ingress/e2e-inf-a-203de8-openshift-default-5f878c54b5-8c99l"
Jul 06 18:49:11.137244 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:11.137178    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/38d03a05-5c33-47fe-ad1d-6866511596bf-istio-envoy\") pod \"e2e-inf-a-203de8-openshift-default-5f878c54b5-8c99l\" (UID: \"38d03a05-5c33-47fe-ad1d-6866511596bf\") " pod="openshift-ingress/e2e-inf-a-203de8-openshift-default-5f878c54b5-8c99l"
Jul 06 18:49:11.237825 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:11.237785    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/38d03a05-5c33-47fe-ad1d-6866511596bf-istio-token\") pod \"e2e-inf-a-203de8-openshift-default-5f878c54b5-8c99l\" (UID: \"38d03a05-5c33-47fe-ad1d-6866511596bf\") " pod="openshift-ingress/e2e-inf-a-203de8-openshift-default-5f878c54b5-8c99l"
Jul 06 18:49:11.237825 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:11.237826    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/38d03a05-5c33-47fe-ad1d-6866511596bf-istio-podinfo\") pod \"e2e-inf-a-203de8-openshift-default-5f878c54b5-8c99l\" (UID: \"38d03a05-5c33-47fe-ad1d-6866511596bf\") " pod="openshift-ingress/e2e-inf-a-203de8-openshift-default-5f878c54b5-8c99l"
Jul 06 18:49:11.238141 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:11.237847    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/38d03a05-5c33-47fe-ad1d-6866511596bf-credential-socket\") pod \"e2e-inf-a-203de8-openshift-default-5f878c54b5-8c99l\" (UID: \"38d03a05-5c33-47fe-ad1d-6866511596bf\") " pod="openshift-ingress/e2e-inf-a-203de8-openshift-default-5f878c54b5-8c99l"
Jul 06 18:49:11.238141 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:11.237876    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/38d03a05-5c33-47fe-ad1d-6866511596bf-istio-envoy\") pod \"e2e-inf-a-203de8-openshift-default-5f878c54b5-8c99l\" (UID: \"38d03a05-5c33-47fe-ad1d-6866511596bf\") " pod="openshift-ingress/e2e-inf-a-203de8-openshift-default-5f878c54b5-8c99l"
Jul 06 18:49:11.238141 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:11.237912    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/38d03a05-5c33-47fe-ad1d-6866511596bf-istio-data\") pod \"e2e-inf-a-203de8-openshift-default-5f878c54b5-8c99l\" (UID: \"38d03a05-5c33-47fe-ad1d-6866511596bf\") " pod="openshift-ingress/e2e-inf-a-203de8-openshift-default-5f878c54b5-8c99l"
Jul 06 18:49:11.238141 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:11.237930    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/38d03a05-5c33-47fe-ad1d-6866511596bf-istiod-ca-cert\") pod \"e2e-inf-a-203de8-openshift-default-5f878c54b5-8c99l\" (UID: \"38d03a05-5c33-47fe-ad1d-6866511596bf\") " pod="openshift-ingress/e2e-inf-a-203de8-openshift-default-5f878c54b5-8c99l"
Jul 06 18:49:11.238141 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:11.237967    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/38d03a05-5c33-47fe-ad1d-6866511596bf-workload-socket\") pod \"e2e-inf-a-203de8-openshift-default-5f878c54b5-8c99l\" (UID: \"38d03a05-5c33-47fe-ad1d-6866511596bf\") " pod="openshift-ingress/e2e-inf-a-203de8-openshift-default-5f878c54b5-8c99l"
Jul 06 18:49:11.238141 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:11.237999    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/38d03a05-5c33-47fe-ad1d-6866511596bf-workload-certs\") pod \"e2e-inf-a-203de8-openshift-default-5f878c54b5-8c99l\" (UID: \"38d03a05-5c33-47fe-ad1d-6866511596bf\") " pod="openshift-ingress/e2e-inf-a-203de8-openshift-default-5f878c54b5-8c99l"
Jul 06 18:49:11.238141 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:11.238055    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pxgnr\" (UniqueName: \"kubernetes.io/projected/38d03a05-5c33-47fe-ad1d-6866511596bf-kube-api-access-pxgnr\") pod \"e2e-inf-a-203de8-openshift-default-5f878c54b5-8c99l\" (UID: \"38d03a05-5c33-47fe-ad1d-6866511596bf\") " pod="openshift-ingress/e2e-inf-a-203de8-openshift-default-5f878c54b5-8c99l"
Jul 06 18:49:11.238580 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:11.238354    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/38d03a05-5c33-47fe-ad1d-6866511596bf-credential-socket\") pod \"e2e-inf-a-203de8-openshift-default-5f878c54b5-8c99l\" (UID: \"38d03a05-5c33-47fe-ad1d-6866511596bf\") " pod="openshift-ingress/e2e-inf-a-203de8-openshift-default-5f878c54b5-8c99l"
Jul 06 18:49:11.238580 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:11.238478    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/38d03a05-5c33-47fe-ad1d-6866511596bf-workload-socket\") pod \"e2e-inf-a-203de8-openshift-default-5f878c54b5-8c99l\" (UID: \"38d03a05-5c33-47fe-ad1d-6866511596bf\") " pod="openshift-ingress/e2e-inf-a-203de8-openshift-default-5f878c54b5-8c99l"
Jul 06 18:49:11.238580 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:11.238521    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/38d03a05-5c33-47fe-ad1d-6866511596bf-istio-data\") pod \"e2e-inf-a-203de8-openshift-default-5f878c54b5-8c99l\" (UID: \"38d03a05-5c33-47fe-ad1d-6866511596bf\") " pod="openshift-ingress/e2e-inf-a-203de8-openshift-default-5f878c54b5-8c99l"
Jul 06 18:49:11.238748 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:11.238658    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/38d03a05-5c33-47fe-ad1d-6866511596bf-istiod-ca-cert\") pod \"e2e-inf-a-203de8-openshift-default-5f878c54b5-8c99l\" (UID: \"38d03a05-5c33-47fe-ad1d-6866511596bf\") " pod="openshift-ingress/e2e-inf-a-203de8-openshift-default-5f878c54b5-8c99l"
Jul 06 18:49:11.238748 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:11.238729    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/38d03a05-5c33-47fe-ad1d-6866511596bf-workload-certs\") pod \"e2e-inf-a-203de8-openshift-default-5f878c54b5-8c99l\" (UID: \"38d03a05-5c33-47fe-ad1d-6866511596bf\") " pod="openshift-ingress/e2e-inf-a-203de8-openshift-default-5f878c54b5-8c99l"
Jul 06 18:49:11.240306 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:11.240286    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/38d03a05-5c33-47fe-ad1d-6866511596bf-istio-envoy\") pod \"e2e-inf-a-203de8-openshift-default-5f878c54b5-8c99l\" (UID: \"38d03a05-5c33-47fe-ad1d-6866511596bf\") " pod="openshift-ingress/e2e-inf-a-203de8-openshift-default-5f878c54b5-8c99l"
Jul 06 18:49:11.240434 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:11.240418    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/38d03a05-5c33-47fe-ad1d-6866511596bf-istio-podinfo\") pod \"e2e-inf-a-203de8-openshift-default-5f878c54b5-8c99l\" (UID: \"38d03a05-5c33-47fe-ad1d-6866511596bf\") " pod="openshift-ingress/e2e-inf-a-203de8-openshift-default-5f878c54b5-8c99l"
Jul 06 18:49:11.248167 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:11.248140    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/38d03a05-5c33-47fe-ad1d-6866511596bf-istio-token\") pod \"e2e-inf-a-203de8-openshift-default-5f878c54b5-8c99l\" (UID: \"38d03a05-5c33-47fe-ad1d-6866511596bf\") " pod="openshift-ingress/e2e-inf-a-203de8-openshift-default-5f878c54b5-8c99l"
Jul 06 18:49:11.248363 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:11.248347    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pxgnr\" (UniqueName: \"kubernetes.io/projected/38d03a05-5c33-47fe-ad1d-6866511596bf-kube-api-access-pxgnr\") pod \"e2e-inf-a-203de8-openshift-default-5f878c54b5-8c99l\" (UID: \"38d03a05-5c33-47fe-ad1d-6866511596bf\") " pod="openshift-ingress/e2e-inf-a-203de8-openshift-default-5f878c54b5-8c99l"
Jul 06 18:49:11.340900 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:11.340863    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-203de8-openshift-default-5f878c54b5-8c99l"
Jul 06 18:49:11.483948 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:11.483805    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-203de8-openshift-default-5f878c54b5-8c99l"]
Jul 06 18:49:11.486199 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:49:11.486164    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38d03a05_5c33_47fe_ad1d_6866511596bf.slice/crio-b797d9041e9dc02a1d4666d478b9af0b19fb75fc1719833904d0af5501e1390a WatchSource:0}: Error finding container b797d9041e9dc02a1d4666d478b9af0b19fb75fc1719833904d0af5501e1390a: Status 404 returned error can't find the container with id b797d9041e9dc02a1d4666d478b9af0b19fb75fc1719833904d0af5501e1390a
Jul 06 18:49:11.488259 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:11.488225    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 06 18:49:11.488340 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:11.488296    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 06 18:49:11.488340 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:11.488327    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 06 18:49:11.880882 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:11.880844    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-203de8-openshift-default-5f878c54b5-8c99l" event={"ID":"38d03a05-5c33-47fe-ad1d-6866511596bf","Type":"ContainerStarted","Data":"34aee067e703743f847d92657b9b8864ca074f3e6977762884aa15ba90a7681b"}
Jul 06 18:49:11.881113 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:11.880903    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-203de8-openshift-default-5f878c54b5-8c99l" event={"ID":"38d03a05-5c33-47fe-ad1d-6866511596bf","Type":"ContainerStarted","Data":"b797d9041e9dc02a1d4666d478b9af0b19fb75fc1719833904d0af5501e1390a"}
Jul 06 18:49:11.920441 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:11.920379    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-203de8-openshift-default-5f878c54b5-8c99l" podStartSLOduration=1.9203568610000001 podStartE2EDuration="1.920356861s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:49:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:49:11.917398909 +0000 UTC m=+1887.074797395" watchObservedRunningTime="2026-07-06 18:49:11.920356861 +0000 UTC m=+1887.077755345"
Jul 06 18:49:12.341494 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:12.341458    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-203de8-openshift-default-5f878c54b5-8c99l"
Jul 06 18:49:12.346595 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:12.346570    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-203de8-openshift-default-5f878c54b5-8c99l"
Jul 06 18:49:12.885464 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:12.885434    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-203de8-openshift-default-5f878c54b5-8c99l"
Jul 06 18:49:12.886429 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:12.886409    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-203de8-openshift-default-5f878c54b5-8c99l"
Jul 06 18:49:14.481228 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:14.481181    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-inf-a-203de8-5f967d57dd-jlftr"]
Jul 06 18:49:14.481725 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:14.481709    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5e20e860-569b-4a59-bd0d-c4bac25f9bd3" containerName="istio-proxy"
Jul 06 18:49:14.481770 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:14.481727    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="5e20e860-569b-4a59-bd0d-c4bac25f9bd3" containerName="istio-proxy"
Jul 06 18:49:14.485275 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:14.485254    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-203de8-5f967d57dd-jlftr"
Jul 06 18:49:14.489993 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:14.489964    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-inf-a-203de8-serving-cert\""
Jul 06 18:49:14.514324 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:14.514289    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-203de8-5f967d57dd-jlftr"]
Jul 06 18:49:14.568670 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:14.568630    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l2mcl\" (UniqueName: \"kubernetes.io/projected/3e7a445c-2e56-4fbe-8e0c-0e38433174f7-kube-api-access-l2mcl\") pod \"maas-api-e2e-inf-a-203de8-5f967d57dd-jlftr\" (UID: \"3e7a445c-2e56-4fbe-8e0c-0e38433174f7\") " pod="opendatahub/maas-api-e2e-inf-a-203de8-5f967d57dd-jlftr"
Jul 06 18:49:14.568840 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:14.568735    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3e7a445c-2e56-4fbe-8e0c-0e38433174f7-maas-api-tls\") pod \"maas-api-e2e-inf-a-203de8-5f967d57dd-jlftr\" (UID: \"3e7a445c-2e56-4fbe-8e0c-0e38433174f7\") " pod="opendatahub/maas-api-e2e-inf-a-203de8-5f967d57dd-jlftr"
Jul 06 18:49:14.568840 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:14.568778    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3e7a445c-2e56-4fbe-8e0c-0e38433174f7-openshift-service-ca\") pod \"maas-api-e2e-inf-a-203de8-5f967d57dd-jlftr\" (UID: \"3e7a445c-2e56-4fbe-8e0c-0e38433174f7\") " pod="opendatahub/maas-api-e2e-inf-a-203de8-5f967d57dd-jlftr"
Jul 06 18:49:14.669833 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:14.669800    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l2mcl\" (UniqueName: \"kubernetes.io/projected/3e7a445c-2e56-4fbe-8e0c-0e38433174f7-kube-api-access-l2mcl\") pod \"maas-api-e2e-inf-a-203de8-5f967d57dd-jlftr\" (UID: \"3e7a445c-2e56-4fbe-8e0c-0e38433174f7\") " pod="opendatahub/maas-api-e2e-inf-a-203de8-5f967d57dd-jlftr"
Jul 06 18:49:14.670042 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:14.669862    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3e7a445c-2e56-4fbe-8e0c-0e38433174f7-maas-api-tls\") pod \"maas-api-e2e-inf-a-203de8-5f967d57dd-jlftr\" (UID: \"3e7a445c-2e56-4fbe-8e0c-0e38433174f7\") " pod="opendatahub/maas-api-e2e-inf-a-203de8-5f967d57dd-jlftr"
Jul 06 18:49:14.670042 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:14.669894    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3e7a445c-2e56-4fbe-8e0c-0e38433174f7-openshift-service-ca\") pod \"maas-api-e2e-inf-a-203de8-5f967d57dd-jlftr\" (UID: \"3e7a445c-2e56-4fbe-8e0c-0e38433174f7\") " pod="opendatahub/maas-api-e2e-inf-a-203de8-5f967d57dd-jlftr"
Jul 06 18:49:14.670495 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:14.670476    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3e7a445c-2e56-4fbe-8e0c-0e38433174f7-openshift-service-ca\") pod \"maas-api-e2e-inf-a-203de8-5f967d57dd-jlftr\" (UID: \"3e7a445c-2e56-4fbe-8e0c-0e38433174f7\") " pod="opendatahub/maas-api-e2e-inf-a-203de8-5f967d57dd-jlftr"
Jul 06 18:49:14.672408 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:14.672388    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3e7a445c-2e56-4fbe-8e0c-0e38433174f7-maas-api-tls\") pod \"maas-api-e2e-inf-a-203de8-5f967d57dd-jlftr\" (UID: \"3e7a445c-2e56-4fbe-8e0c-0e38433174f7\") " pod="opendatahub/maas-api-e2e-inf-a-203de8-5f967d57dd-jlftr"
Jul 06 18:49:14.701300 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:14.701268    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l2mcl\" (UniqueName: \"kubernetes.io/projected/3e7a445c-2e56-4fbe-8e0c-0e38433174f7-kube-api-access-l2mcl\") pod \"maas-api-e2e-inf-a-203de8-5f967d57dd-jlftr\" (UID: \"3e7a445c-2e56-4fbe-8e0c-0e38433174f7\") " pod="opendatahub/maas-api-e2e-inf-a-203de8-5f967d57dd-jlftr"
Jul 06 18:49:14.796417 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:14.796324    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-203de8-5f967d57dd-jlftr"
Jul 06 18:49:14.959781 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:14.959751    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-203de8-5f967d57dd-jlftr"]
Jul 06 18:49:14.960735 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:49:14.960710    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e7a445c_2e56_4fbe_8e0c_0e38433174f7.slice/crio-06fdf366cfbf01cfe0afc015ea783057cc89504b722291214126c550a7667229 WatchSource:0}: Error finding container 06fdf366cfbf01cfe0afc015ea783057cc89504b722291214126c550a7667229: Status 404 returned error can't find the container with id 06fdf366cfbf01cfe0afc015ea783057cc89504b722291214126c550a7667229
Jul 06 18:49:14.962147 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:14.962129    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 06 18:49:15.902324 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:15.902285    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-203de8-5f967d57dd-jlftr" event={"ID":"3e7a445c-2e56-4fbe-8e0c-0e38433174f7","Type":"ContainerStarted","Data":"b65b2f9af0c12db13cf15d9eca69a77ea6939200fa67c4388892a966911ad498"}
Jul 06 18:49:15.902768 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:15.902338    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-203de8-5f967d57dd-jlftr" event={"ID":"3e7a445c-2e56-4fbe-8e0c-0e38433174f7","Type":"ContainerStarted","Data":"06fdf366cfbf01cfe0afc015ea783057cc89504b722291214126c550a7667229"}
Jul 06 18:49:15.902768 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:15.902377    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-inf-a-203de8-5f967d57dd-jlftr"
Jul 06 18:49:15.948964 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:15.948908    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-inf-a-203de8-5f967d57dd-jlftr" podStartSLOduration=1.600031781 podStartE2EDuration="1.948878208s" totalImagesPullingTime="348.846427ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:49:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:49:15.947639309 +0000 UTC m=+1891.105037808" watchObservedRunningTime="2026-07-06 18:49:15.948878208 +0000 UTC m=+1891.106276695"
Jul 06 18:49:19.463991 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:19.463953    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-aea86e-openshift-default-6ff5849847-fg5tb"]
Jul 06 18:49:19.467285 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:19.467268    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-aea86e-openshift-default-6ff5849847-fg5tb"
Jul 06 18:49:19.471120 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:19.471097    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-aea86e-openshift-default-dockercfg-rvt2m\""
Jul 06 18:49:19.512796 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:19.512763    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-aea86e-openshift-default-6ff5849847-fg5tb"]
Jul 06 18:49:19.513225 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:19.513196    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2099861f-1605-478e-99ac-224ce565f19c-istio-token\") pod \"e2e-inf-b-aea86e-openshift-default-6ff5849847-fg5tb\" (UID: \"2099861f-1605-478e-99ac-224ce565f19c\") " pod="openshift-ingress/e2e-inf-b-aea86e-openshift-default-6ff5849847-fg5tb"
Jul 06 18:49:19.513337 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:19.513240    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2099861f-1605-478e-99ac-224ce565f19c-istiod-ca-cert\") pod \"e2e-inf-b-aea86e-openshift-default-6ff5849847-fg5tb\" (UID: \"2099861f-1605-478e-99ac-224ce565f19c\") " pod="openshift-ingress/e2e-inf-b-aea86e-openshift-default-6ff5849847-fg5tb"
Jul 06 18:49:19.513337 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:19.513277    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fdsd5\" (UniqueName: \"kubernetes.io/projected/2099861f-1605-478e-99ac-224ce565f19c-kube-api-access-fdsd5\") pod \"e2e-inf-b-aea86e-openshift-default-6ff5849847-fg5tb\" (UID: \"2099861f-1605-478e-99ac-224ce565f19c\") " pod="openshift-ingress/e2e-inf-b-aea86e-openshift-default-6ff5849847-fg5tb"
Jul 06 18:49:19.513337 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:19.513311    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2099861f-1605-478e-99ac-224ce565f19c-istio-envoy\") pod \"e2e-inf-b-aea86e-openshift-default-6ff5849847-fg5tb\" (UID: \"2099861f-1605-478e-99ac-224ce565f19c\") " pod="openshift-ingress/e2e-inf-b-aea86e-openshift-default-6ff5849847-fg5tb"
Jul 06 18:49:19.513451 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:19.513387    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2099861f-1605-478e-99ac-224ce565f19c-workload-certs\") pod \"e2e-inf-b-aea86e-openshift-default-6ff5849847-fg5tb\" (UID: \"2099861f-1605-478e-99ac-224ce565f19c\") " pod="openshift-ingress/e2e-inf-b-aea86e-openshift-default-6ff5849847-fg5tb"
Jul 06 18:49:19.513451 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:19.513425    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2099861f-1605-478e-99ac-224ce565f19c-istio-podinfo\") pod \"e2e-inf-b-aea86e-openshift-default-6ff5849847-fg5tb\" (UID: \"2099861f-1605-478e-99ac-224ce565f19c\") " pod="openshift-ingress/e2e-inf-b-aea86e-openshift-default-6ff5849847-fg5tb"
Jul 06 18:49:19.513560 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:19.513538    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2099861f-1605-478e-99ac-224ce565f19c-workload-socket\") pod \"e2e-inf-b-aea86e-openshift-default-6ff5849847-fg5tb\" (UID: \"2099861f-1605-478e-99ac-224ce565f19c\") " pod="openshift-ingress/e2e-inf-b-aea86e-openshift-default-6ff5849847-fg5tb"
Jul 06 18:49:19.513596 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:19.513575    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2099861f-1605-478e-99ac-224ce565f19c-credential-socket\") pod \"e2e-inf-b-aea86e-openshift-default-6ff5849847-fg5tb\" (UID: \"2099861f-1605-478e-99ac-224ce565f19c\") " pod="openshift-ingress/e2e-inf-b-aea86e-openshift-default-6ff5849847-fg5tb"
Jul 06 18:49:19.513634 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:19.513593    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2099861f-1605-478e-99ac-224ce565f19c-istio-data\") pod \"e2e-inf-b-aea86e-openshift-default-6ff5849847-fg5tb\" (UID: \"2099861f-1605-478e-99ac-224ce565f19c\") " pod="openshift-ingress/e2e-inf-b-aea86e-openshift-default-6ff5849847-fg5tb"
Jul 06 18:49:19.614879 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:19.614836    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2099861f-1605-478e-99ac-224ce565f19c-istio-token\") pod \"e2e-inf-b-aea86e-openshift-default-6ff5849847-fg5tb\" (UID: \"2099861f-1605-478e-99ac-224ce565f19c\") " pod="openshift-ingress/e2e-inf-b-aea86e-openshift-default-6ff5849847-fg5tb"
Jul 06 18:49:19.614879 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:19.614876    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2099861f-1605-478e-99ac-224ce565f19c-istiod-ca-cert\") pod \"e2e-inf-b-aea86e-openshift-default-6ff5849847-fg5tb\" (UID: \"2099861f-1605-478e-99ac-224ce565f19c\") " pod="openshift-ingress/e2e-inf-b-aea86e-openshift-default-6ff5849847-fg5tb"
Jul 06 18:49:19.615176 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:19.614901    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fdsd5\" (UniqueName: \"kubernetes.io/projected/2099861f-1605-478e-99ac-224ce565f19c-kube-api-access-fdsd5\") pod \"e2e-inf-b-aea86e-openshift-default-6ff5849847-fg5tb\" (UID: \"2099861f-1605-478e-99ac-224ce565f19c\") " pod="openshift-ingress/e2e-inf-b-aea86e-openshift-default-6ff5849847-fg5tb"
Jul 06 18:49:19.615176 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:19.614925    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2099861f-1605-478e-99ac-224ce565f19c-istio-envoy\") pod \"e2e-inf-b-aea86e-openshift-default-6ff5849847-fg5tb\" (UID: \"2099861f-1605-478e-99ac-224ce565f19c\") " pod="openshift-ingress/e2e-inf-b-aea86e-openshift-default-6ff5849847-fg5tb"
Jul 06 18:49:19.615176 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:19.614955    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2099861f-1605-478e-99ac-224ce565f19c-workload-certs\") pod \"e2e-inf-b-aea86e-openshift-default-6ff5849847-fg5tb\" (UID: \"2099861f-1605-478e-99ac-224ce565f19c\") " pod="openshift-ingress/e2e-inf-b-aea86e-openshift-default-6ff5849847-fg5tb"
Jul 06 18:49:19.615176 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:19.614979    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2099861f-1605-478e-99ac-224ce565f19c-istio-podinfo\") pod \"e2e-inf-b-aea86e-openshift-default-6ff5849847-fg5tb\" (UID: \"2099861f-1605-478e-99ac-224ce565f19c\") " pod="openshift-ingress/e2e-inf-b-aea86e-openshift-default-6ff5849847-fg5tb"
Jul 06 18:49:19.615176 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:19.615090    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2099861f-1605-478e-99ac-224ce565f19c-workload-socket\") pod \"e2e-inf-b-aea86e-openshift-default-6ff5849847-fg5tb\" (UID: \"2099861f-1605-478e-99ac-224ce565f19c\") " pod="openshift-ingress/e2e-inf-b-aea86e-openshift-default-6ff5849847-fg5tb"
Jul 06 18:49:19.615176 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:19.615118    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2099861f-1605-478e-99ac-224ce565f19c-credential-socket\") pod \"e2e-inf-b-aea86e-openshift-default-6ff5849847-fg5tb\" (UID: \"2099861f-1605-478e-99ac-224ce565f19c\") " pod="openshift-ingress/e2e-inf-b-aea86e-openshift-default-6ff5849847-fg5tb"
Jul 06 18:49:19.615176 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:19.615143    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2099861f-1605-478e-99ac-224ce565f19c-istio-data\") pod \"e2e-inf-b-aea86e-openshift-default-6ff5849847-fg5tb\" (UID: \"2099861f-1605-478e-99ac-224ce565f19c\") " pod="openshift-ingress/e2e-inf-b-aea86e-openshift-default-6ff5849847-fg5tb"
Jul 06 18:49:19.615540 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:19.615422    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2099861f-1605-478e-99ac-224ce565f19c-workload-certs\") pod \"e2e-inf-b-aea86e-openshift-default-6ff5849847-fg5tb\" (UID: \"2099861f-1605-478e-99ac-224ce565f19c\") " pod="openshift-ingress/e2e-inf-b-aea86e-openshift-default-6ff5849847-fg5tb"
Jul 06 18:49:19.615640 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:19.615613    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2099861f-1605-478e-99ac-224ce565f19c-credential-socket\") pod \"e2e-inf-b-aea86e-openshift-default-6ff5849847-fg5tb\" (UID: \"2099861f-1605-478e-99ac-224ce565f19c\") " pod="openshift-ingress/e2e-inf-b-aea86e-openshift-default-6ff5849847-fg5tb"
Jul 06 18:49:19.615708 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:19.615662    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2099861f-1605-478e-99ac-224ce565f19c-workload-socket\") pod \"e2e-inf-b-aea86e-openshift-default-6ff5849847-fg5tb\" (UID: \"2099861f-1605-478e-99ac-224ce565f19c\") " pod="openshift-ingress/e2e-inf-b-aea86e-openshift-default-6ff5849847-fg5tb"
Jul 06 18:49:19.615765 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:19.615716    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2099861f-1605-478e-99ac-224ce565f19c-istiod-ca-cert\") pod \"e2e-inf-b-aea86e-openshift-default-6ff5849847-fg5tb\" (UID: \"2099861f-1605-478e-99ac-224ce565f19c\") " pod="openshift-ingress/e2e-inf-b-aea86e-openshift-default-6ff5849847-fg5tb"
Jul 06 18:49:19.615819 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:19.615761    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2099861f-1605-478e-99ac-224ce565f19c-istio-data\") pod \"e2e-inf-b-aea86e-openshift-default-6ff5849847-fg5tb\" (UID: \"2099861f-1605-478e-99ac-224ce565f19c\") " pod="openshift-ingress/e2e-inf-b-aea86e-openshift-default-6ff5849847-fg5tb"
Jul 06 18:49:19.617239 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:19.617220    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2099861f-1605-478e-99ac-224ce565f19c-istio-envoy\") pod \"e2e-inf-b-aea86e-openshift-default-6ff5849847-fg5tb\" (UID: \"2099861f-1605-478e-99ac-224ce565f19c\") " pod="openshift-ingress/e2e-inf-b-aea86e-openshift-default-6ff5849847-fg5tb"
Jul 06 18:49:19.617699 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:19.617678    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2099861f-1605-478e-99ac-224ce565f19c-istio-podinfo\") pod \"e2e-inf-b-aea86e-openshift-default-6ff5849847-fg5tb\" (UID: \"2099861f-1605-478e-99ac-224ce565f19c\") " pod="openshift-ingress/e2e-inf-b-aea86e-openshift-default-6ff5849847-fg5tb"
Jul 06 18:49:19.630337 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:19.630308    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fdsd5\" (UniqueName: \"kubernetes.io/projected/2099861f-1605-478e-99ac-224ce565f19c-kube-api-access-fdsd5\") pod \"e2e-inf-b-aea86e-openshift-default-6ff5849847-fg5tb\" (UID: \"2099861f-1605-478e-99ac-224ce565f19c\") " pod="openshift-ingress/e2e-inf-b-aea86e-openshift-default-6ff5849847-fg5tb"
Jul 06 18:49:19.630471 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:19.630452    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2099861f-1605-478e-99ac-224ce565f19c-istio-token\") pod \"e2e-inf-b-aea86e-openshift-default-6ff5849847-fg5tb\" (UID: \"2099861f-1605-478e-99ac-224ce565f19c\") " pod="openshift-ingress/e2e-inf-b-aea86e-openshift-default-6ff5849847-fg5tb"
Jul 06 18:49:19.778367 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:19.778263    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-aea86e-openshift-default-6ff5849847-fg5tb"
Jul 06 18:49:19.946508 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:19.946478    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-aea86e-openshift-default-6ff5849847-fg5tb"]
Jul 06 18:49:19.947489 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:49:19.947464    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2099861f_1605_478e_99ac_224ce565f19c.slice/crio-cf830b1db60f73f518ad9cdba112e1e9f15eee51e2be3def3c0934e5cce032fd WatchSource:0}: Error finding container cf830b1db60f73f518ad9cdba112e1e9f15eee51e2be3def3c0934e5cce032fd: Status 404 returned error can't find the container with id cf830b1db60f73f518ad9cdba112e1e9f15eee51e2be3def3c0934e5cce032fd
Jul 06 18:49:19.949720 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:19.949689    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 06 18:49:19.949814 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:19.949755    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 06 18:49:19.949814 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:19.949783    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 06 18:49:20.934361 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:20.934326    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-aea86e-openshift-default-6ff5849847-fg5tb" event={"ID":"2099861f-1605-478e-99ac-224ce565f19c","Type":"ContainerStarted","Data":"0f85ee3e077f4988135316f02809516ec171132fac169687f158781fb58b3e13"}
Jul 06 18:49:20.934779 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:20.934420    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-aea86e-openshift-default-6ff5849847-fg5tb" event={"ID":"2099861f-1605-478e-99ac-224ce565f19c","Type":"ContainerStarted","Data":"cf830b1db60f73f518ad9cdba112e1e9f15eee51e2be3def3c0934e5cce032fd"}
Jul 06 18:49:21.779102 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:21.779058    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-aea86e-openshift-default-6ff5849847-fg5tb"
Jul 06 18:49:21.783496 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:21.783469    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-aea86e-openshift-default-6ff5849847-fg5tb"
Jul 06 18:49:21.829592 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:21.829537    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-aea86e-openshift-default-6ff5849847-fg5tb" podStartSLOduration=2.8295212149999998 podStartE2EDuration="2.829521215s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:49:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:49:20.973872188 +0000 UTC m=+1896.131270674" watchObservedRunningTime="2026-07-06 18:49:21.829521215 +0000 UTC m=+1896.986919703"
Jul 06 18:49:21.912150 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:21.912115    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-inf-a-203de8-5f967d57dd-jlftr"
Jul 06 18:49:21.939058 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:21.939026    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-aea86e-openshift-default-6ff5849847-fg5tb"
Jul 06 18:49:21.939895 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:21.939874    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-aea86e-openshift-default-6ff5849847-fg5tb"
Jul 06 18:49:23.404068 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:23.404029    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-inf-b-aea86e-5b9bc9fdc9-bvbn6"]
Jul 06 18:49:23.407441 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:23.407417    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-aea86e-5b9bc9fdc9-bvbn6"
Jul 06 18:49:23.410746 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:23.410720    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-inf-b-aea86e-serving-cert\""
Jul 06 18:49:23.461237 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:23.461198    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-aea86e-5b9bc9fdc9-bvbn6"]
Jul 06 18:49:23.557612 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:23.557574    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/829713c7-f3f2-4f5a-b4b8-44d840743dae-maas-api-tls\") pod \"maas-api-e2e-inf-b-aea86e-5b9bc9fdc9-bvbn6\" (UID: \"829713c7-f3f2-4f5a-b4b8-44d840743dae\") " pod="opendatahub/maas-api-e2e-inf-b-aea86e-5b9bc9fdc9-bvbn6"
Jul 06 18:49:23.557825 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:23.557640    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/829713c7-f3f2-4f5a-b4b8-44d840743dae-openshift-service-ca\") pod \"maas-api-e2e-inf-b-aea86e-5b9bc9fdc9-bvbn6\" (UID: \"829713c7-f3f2-4f5a-b4b8-44d840743dae\") " pod="opendatahub/maas-api-e2e-inf-b-aea86e-5b9bc9fdc9-bvbn6"
Jul 06 18:49:23.557825 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:23.557784    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h59gd\" (UniqueName: \"kubernetes.io/projected/829713c7-f3f2-4f5a-b4b8-44d840743dae-kube-api-access-h59gd\") pod \"maas-api-e2e-inf-b-aea86e-5b9bc9fdc9-bvbn6\" (UID: \"829713c7-f3f2-4f5a-b4b8-44d840743dae\") " pod="opendatahub/maas-api-e2e-inf-b-aea86e-5b9bc9fdc9-bvbn6"
Jul 06 18:49:23.659060 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:23.658920    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/829713c7-f3f2-4f5a-b4b8-44d840743dae-openshift-service-ca\") pod \"maas-api-e2e-inf-b-aea86e-5b9bc9fdc9-bvbn6\" (UID: \"829713c7-f3f2-4f5a-b4b8-44d840743dae\") " pod="opendatahub/maas-api-e2e-inf-b-aea86e-5b9bc9fdc9-bvbn6"
Jul 06 18:49:23.659255 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:23.659060    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h59gd\" (UniqueName: \"kubernetes.io/projected/829713c7-f3f2-4f5a-b4b8-44d840743dae-kube-api-access-h59gd\") pod \"maas-api-e2e-inf-b-aea86e-5b9bc9fdc9-bvbn6\" (UID: \"829713c7-f3f2-4f5a-b4b8-44d840743dae\") " pod="opendatahub/maas-api-e2e-inf-b-aea86e-5b9bc9fdc9-bvbn6"
Jul 06 18:49:23.659255 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:23.659095    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/829713c7-f3f2-4f5a-b4b8-44d840743dae-maas-api-tls\") pod \"maas-api-e2e-inf-b-aea86e-5b9bc9fdc9-bvbn6\" (UID: \"829713c7-f3f2-4f5a-b4b8-44d840743dae\") " pod="opendatahub/maas-api-e2e-inf-b-aea86e-5b9bc9fdc9-bvbn6"
Jul 06 18:49:23.659623 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:23.659599    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/829713c7-f3f2-4f5a-b4b8-44d840743dae-openshift-service-ca\") pod \"maas-api-e2e-inf-b-aea86e-5b9bc9fdc9-bvbn6\" (UID: \"829713c7-f3f2-4f5a-b4b8-44d840743dae\") " pod="opendatahub/maas-api-e2e-inf-b-aea86e-5b9bc9fdc9-bvbn6"
Jul 06 18:49:23.661450 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:23.661428    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/829713c7-f3f2-4f5a-b4b8-44d840743dae-maas-api-tls\") pod \"maas-api-e2e-inf-b-aea86e-5b9bc9fdc9-bvbn6\" (UID: \"829713c7-f3f2-4f5a-b4b8-44d840743dae\") " pod="opendatahub/maas-api-e2e-inf-b-aea86e-5b9bc9fdc9-bvbn6"
Jul 06 18:49:23.673950 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:23.673919    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h59gd\" (UniqueName: \"kubernetes.io/projected/829713c7-f3f2-4f5a-b4b8-44d840743dae-kube-api-access-h59gd\") pod \"maas-api-e2e-inf-b-aea86e-5b9bc9fdc9-bvbn6\" (UID: \"829713c7-f3f2-4f5a-b4b8-44d840743dae\") " pod="opendatahub/maas-api-e2e-inf-b-aea86e-5b9bc9fdc9-bvbn6"
Jul 06 18:49:23.718458 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:23.718425    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-aea86e-5b9bc9fdc9-bvbn6"
Jul 06 18:49:23.922088 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:23.922061    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-aea86e-5b9bc9fdc9-bvbn6"]
Jul 06 18:49:23.922469 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:49:23.922447    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod829713c7_f3f2_4f5a_b4b8_44d840743dae.slice/crio-0d1c577ab8d85f52b96099a0214eeca0b9a35c5efac7ae571634691b2ef3bb56 WatchSource:0}: Error finding container 0d1c577ab8d85f52b96099a0214eeca0b9a35c5efac7ae571634691b2ef3bb56: Status 404 returned error can't find the container with id 0d1c577ab8d85f52b96099a0214eeca0b9a35c5efac7ae571634691b2ef3bb56
Jul 06 18:49:23.958210 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:23.958164    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-aea86e-5b9bc9fdc9-bvbn6" event={"ID":"829713c7-f3f2-4f5a-b4b8-44d840743dae","Type":"ContainerStarted","Data":"0d1c577ab8d85f52b96099a0214eeca0b9a35c5efac7ae571634691b2ef3bb56"}
Jul 06 18:49:24.963907 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:24.963872    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-aea86e-5b9bc9fdc9-bvbn6" event={"ID":"829713c7-f3f2-4f5a-b4b8-44d840743dae","Type":"ContainerStarted","Data":"c5f61cd64c2d2b71cca6928d066f7fe1f1dfd0d735498f292592799b80d3cc02"}
Jul 06 18:49:24.964340 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:24.964002    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-inf-b-aea86e-5b9bc9fdc9-bvbn6"
Jul 06 18:49:28.927281 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:28.927239    2580 patch_prober.go:28] interesting pod/e2e-inf-a-203de8-openshift-default-5f878c54b5-8c99l container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 06 18:49:28.927961 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:28.927917    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-203de8-openshift-default-5f878c54b5-8c99l" podUID="38d03a05-5c33-47fe-ad1d-6866511596bf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 06 18:49:29.991575 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:29.991538    2580 generic.go:356] "Generic (PLEG): container finished" podID="38d03a05-5c33-47fe-ad1d-6866511596bf" containerID="34aee067e703743f847d92657b9b8864ca074f3e6977762884aa15ba90a7681b" exitCode=0
Jul 06 18:49:29.992036 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:29.991611    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-203de8-openshift-default-5f878c54b5-8c99l" event={"ID":"38d03a05-5c33-47fe-ad1d-6866511596bf","Type":"ContainerDied","Data":"34aee067e703743f847d92657b9b8864ca074f3e6977762884aa15ba90a7681b"}
Jul 06 18:49:29.992036 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:29.992028    2580 scope.go:117] "RemoveContainer" containerID="34aee067e703743f847d92657b9b8864ca074f3e6977762884aa15ba90a7681b"
Jul 06 18:49:29.993003 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:29.992976    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 06 18:49:29.993084 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:29.993073    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 06 18:49:29.993121 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:29.993107    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 06 18:49:30.016229 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:30.016175    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-inf-b-aea86e-5b9bc9fdc9-bvbn6" podStartSLOduration=6.593761898 podStartE2EDuration="7.016158955s" totalImagesPullingTime="422.397057ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:49:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:49:25.038778589 +0000 UTC m=+1900.196177078" watchObservedRunningTime="2026-07-06 18:49:30.016158955 +0000 UTC m=+1905.173557444"
Jul 06 18:49:30.981269 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:30.981153    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-inf-b-aea86e-5b9bc9fdc9-bvbn6"
Jul 06 18:49:31.001952 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:31.001898    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-203de8-openshift-default-5f878c54b5-8c99l" event={"ID":"38d03a05-5c33-47fe-ad1d-6866511596bf","Type":"ContainerStarted","Data":"391b5d623ff0ace4dbe037fd398b56b9643d7b169dc253f11f630de201ce1bd9"}
Jul 06 18:49:31.341892 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:31.341844    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-203de8-openshift-default-5f878c54b5-8c99l"
Jul 06 18:49:31.342138 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:31.341982    2580 patch_prober.go:28] interesting pod/e2e-inf-a-203de8-openshift-default-5f878c54b5-8c99l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body=
Jul 06 18:49:31.342138 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:31.342051    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-203de8-openshift-default-5f878c54b5-8c99l" podUID="38d03a05-5c33-47fe-ad1d-6866511596bf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused"
Jul 06 18:49:32.341425 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:32.341375    2580 patch_prober.go:28] interesting pod/e2e-inf-a-203de8-openshift-default-5f878c54b5-8c99l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body=
Jul 06 18:49:32.341979 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:32.341464    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-203de8-openshift-default-5f878c54b5-8c99l" podUID="38d03a05-5c33-47fe-ad1d-6866511596bf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused"
Jul 06 18:49:33.341727 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:33.341687    2580 patch_prober.go:28] interesting pod/e2e-inf-a-203de8-openshift-default-5f878c54b5-8c99l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body=
Jul 06 18:49:33.342359 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:33.342324    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-203de8-openshift-default-5f878c54b5-8c99l" podUID="38d03a05-5c33-47fe-ad1d-6866511596bf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused"
Jul 06 18:49:33.915881 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:33.915845    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-203de8/test-model-203de8-kserve-6d79f8dd8f-zf8b4"]
Jul 06 18:49:33.920154 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:33.920124    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-203de8/test-model-203de8-kserve-6d79f8dd8f-zf8b4"
Jul 06 18:49:33.923256 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:33.923225    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-203de8\"/\"openshift-service-ca.crt\""
Jul 06 18:49:33.923403 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:33.923330    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-203de8\"/\"kube-root-ca.crt\""
Jul 06 18:49:33.924531 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:33.924514    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-203de8\"/\"default-dockercfg-s6zm5\""
Jul 06 18:49:33.924687 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:33.924535    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-203de8\"/\"test-model-203de8-kserve-self-signed-certs\""
Jul 06 18:49:33.949039 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:33.945859    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-203de8/test-model-203de8-kserve-6d79f8dd8f-zf8b4"]
Jul 06 18:49:34.066354 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:34.066321    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a6a5937a-88d6-486a-8e59-3676ef7966e7-home\") pod \"test-model-203de8-kserve-6d79f8dd8f-zf8b4\" (UID: \"a6a5937a-88d6-486a-8e59-3676ef7966e7\") " pod="ai-tenant-e2e-inf-a-203de8/test-model-203de8-kserve-6d79f8dd8f-zf8b4"
Jul 06 18:49:34.066531 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:34.066391    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a6a5937a-88d6-486a-8e59-3676ef7966e7-tls-certs\") pod \"test-model-203de8-kserve-6d79f8dd8f-zf8b4\" (UID: \"a6a5937a-88d6-486a-8e59-3676ef7966e7\") " pod="ai-tenant-e2e-inf-a-203de8/test-model-203de8-kserve-6d79f8dd8f-zf8b4"
Jul 06 18:49:34.066531 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:34.066459    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a6a5937a-88d6-486a-8e59-3676ef7966e7-tmp-dir\") pod \"test-model-203de8-kserve-6d79f8dd8f-zf8b4\" (UID: \"a6a5937a-88d6-486a-8e59-3676ef7966e7\") " pod="ai-tenant-e2e-inf-a-203de8/test-model-203de8-kserve-6d79f8dd8f-zf8b4"
Jul 06 18:49:34.066656 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:34.066564    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tjc9t\" (UniqueName: \"kubernetes.io/projected/a6a5937a-88d6-486a-8e59-3676ef7966e7-kube-api-access-tjc9t\") pod \"test-model-203de8-kserve-6d79f8dd8f-zf8b4\" (UID: \"a6a5937a-88d6-486a-8e59-3676ef7966e7\") " pod="ai-tenant-e2e-inf-a-203de8/test-model-203de8-kserve-6d79f8dd8f-zf8b4"
Jul 06 18:49:34.066700 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:34.066656    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a6a5937a-88d6-486a-8e59-3676ef7966e7-dshm\") pod \"test-model-203de8-kserve-6d79f8dd8f-zf8b4\" (UID: \"a6a5937a-88d6-486a-8e59-3676ef7966e7\") " pod="ai-tenant-e2e-inf-a-203de8/test-model-203de8-kserve-6d79f8dd8f-zf8b4"
Jul 06 18:49:34.066700 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:34.066679    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a6a5937a-88d6-486a-8e59-3676ef7966e7-model-cache\") pod \"test-model-203de8-kserve-6d79f8dd8f-zf8b4\" (UID: \"a6a5937a-88d6-486a-8e59-3676ef7966e7\") " pod="ai-tenant-e2e-inf-a-203de8/test-model-203de8-kserve-6d79f8dd8f-zf8b4"
Jul 06 18:49:34.168147 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:34.168046    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a6a5937a-88d6-486a-8e59-3676ef7966e7-home\") pod \"test-model-203de8-kserve-6d79f8dd8f-zf8b4\" (UID: \"a6a5937a-88d6-486a-8e59-3676ef7966e7\") " pod="ai-tenant-e2e-inf-a-203de8/test-model-203de8-kserve-6d79f8dd8f-zf8b4"
Jul 06 18:49:34.168327 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:34.168169    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a6a5937a-88d6-486a-8e59-3676ef7966e7-tls-certs\") pod \"test-model-203de8-kserve-6d79f8dd8f-zf8b4\" (UID: \"a6a5937a-88d6-486a-8e59-3676ef7966e7\") " pod="ai-tenant-e2e-inf-a-203de8/test-model-203de8-kserve-6d79f8dd8f-zf8b4"
Jul 06 18:49:34.168327 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:34.168201    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a6a5937a-88d6-486a-8e59-3676ef7966e7-tmp-dir\") pod \"test-model-203de8-kserve-6d79f8dd8f-zf8b4\" (UID: \"a6a5937a-88d6-486a-8e59-3676ef7966e7\") " pod="ai-tenant-e2e-inf-a-203de8/test-model-203de8-kserve-6d79f8dd8f-zf8b4"
Jul 06 18:49:34.168327 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:34.168227    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tjc9t\" (UniqueName: \"kubernetes.io/projected/a6a5937a-88d6-486a-8e59-3676ef7966e7-kube-api-access-tjc9t\") pod \"test-model-203de8-kserve-6d79f8dd8f-zf8b4\" (UID: \"a6a5937a-88d6-486a-8e59-3676ef7966e7\") " pod="ai-tenant-e2e-inf-a-203de8/test-model-203de8-kserve-6d79f8dd8f-zf8b4"
Jul 06 18:49:34.168327 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:34.168284    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a6a5937a-88d6-486a-8e59-3676ef7966e7-dshm\") pod \"test-model-203de8-kserve-6d79f8dd8f-zf8b4\" (UID: \"a6a5937a-88d6-486a-8e59-3676ef7966e7\") " pod="ai-tenant-e2e-inf-a-203de8/test-model-203de8-kserve-6d79f8dd8f-zf8b4"
Jul 06 18:49:34.168327 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:34.168308    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a6a5937a-88d6-486a-8e59-3676ef7966e7-model-cache\") pod \"test-model-203de8-kserve-6d79f8dd8f-zf8b4\" (UID: \"a6a5937a-88d6-486a-8e59-3676ef7966e7\") " pod="ai-tenant-e2e-inf-a-203de8/test-model-203de8-kserve-6d79f8dd8f-zf8b4"
Jul 06 18:49:34.168620 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:34.168584    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a6a5937a-88d6-486a-8e59-3676ef7966e7-home\") pod \"test-model-203de8-kserve-6d79f8dd8f-zf8b4\" (UID: \"a6a5937a-88d6-486a-8e59-3676ef7966e7\") " pod="ai-tenant-e2e-inf-a-203de8/test-model-203de8-kserve-6d79f8dd8f-zf8b4"
Jul 06 18:49:34.168691 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:34.168636    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a6a5937a-88d6-486a-8e59-3676ef7966e7-tmp-dir\") pod \"test-model-203de8-kserve-6d79f8dd8f-zf8b4\" (UID: \"a6a5937a-88d6-486a-8e59-3676ef7966e7\") " pod="ai-tenant-e2e-inf-a-203de8/test-model-203de8-kserve-6d79f8dd8f-zf8b4"
Jul 06 18:49:34.168799 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:34.168783    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a6a5937a-88d6-486a-8e59-3676ef7966e7-model-cache\") pod \"test-model-203de8-kserve-6d79f8dd8f-zf8b4\" (UID: \"a6a5937a-88d6-486a-8e59-3676ef7966e7\") " pod="ai-tenant-e2e-inf-a-203de8/test-model-203de8-kserve-6d79f8dd8f-zf8b4"
Jul 06 18:49:34.170585 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:34.170554    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a6a5937a-88d6-486a-8e59-3676ef7966e7-dshm\") pod \"test-model-203de8-kserve-6d79f8dd8f-zf8b4\" (UID: \"a6a5937a-88d6-486a-8e59-3676ef7966e7\") " pod="ai-tenant-e2e-inf-a-203de8/test-model-203de8-kserve-6d79f8dd8f-zf8b4"
Jul 06 18:49:34.170853 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:34.170831    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a6a5937a-88d6-486a-8e59-3676ef7966e7-tls-certs\") pod \"test-model-203de8-kserve-6d79f8dd8f-zf8b4\" (UID: \"a6a5937a-88d6-486a-8e59-3676ef7966e7\") " pod="ai-tenant-e2e-inf-a-203de8/test-model-203de8-kserve-6d79f8dd8f-zf8b4"
Jul 06 18:49:34.177630 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:34.177599    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tjc9t\" (UniqueName: \"kubernetes.io/projected/a6a5937a-88d6-486a-8e59-3676ef7966e7-kube-api-access-tjc9t\") pod \"test-model-203de8-kserve-6d79f8dd8f-zf8b4\" (UID: \"a6a5937a-88d6-486a-8e59-3676ef7966e7\") " pod="ai-tenant-e2e-inf-a-203de8/test-model-203de8-kserve-6d79f8dd8f-zf8b4"
Jul 06 18:49:34.258836 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:34.258791    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-203de8/test-model-203de8-kserve-6d79f8dd8f-zf8b4"
Jul 06 18:49:34.346050 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:34.345704    2580 patch_prober.go:28] interesting pod/e2e-inf-a-203de8-openshift-default-5f878c54b5-8c99l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body=
Jul 06 18:49:34.346050 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:34.345807    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-203de8-openshift-default-5f878c54b5-8c99l" podUID="38d03a05-5c33-47fe-ad1d-6866511596bf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused"
Jul 06 18:49:34.443108 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:34.431949    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-203de8/test-model-203de8-kserve-6d79f8dd8f-zf8b4"]
Jul 06 18:49:34.452697 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:49:34.452647    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda6a5937a_88d6_486a_8e59_3676ef7966e7.slice/crio-0a56a81dbcbe42d6cd182dc773924b8f050d66bcc124fd82c757e39dde95d5dd WatchSource:0}: Error finding container 0a56a81dbcbe42d6cd182dc773924b8f050d66bcc124fd82c757e39dde95d5dd: Status 404 returned error can't find the container with id 0a56a81dbcbe42d6cd182dc773924b8f050d66bcc124fd82c757e39dde95d5dd
Jul 06 18:49:35.024197 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:35.024159    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-203de8/test-model-203de8-kserve-6d79f8dd8f-zf8b4" event={"ID":"a6a5937a-88d6-486a-8e59-3676ef7966e7","Type":"ContainerStarted","Data":"ba2fe5d220c15088615854147c149e4299a59a381a24c7554bd09943c7a71e66"}
Jul 06 18:49:35.024390 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:35.024205    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-203de8/test-model-203de8-kserve-6d79f8dd8f-zf8b4" event={"ID":"a6a5937a-88d6-486a-8e59-3676ef7966e7","Type":"ContainerStarted","Data":"0a56a81dbcbe42d6cd182dc773924b8f050d66bcc124fd82c757e39dde95d5dd"}
Jul 06 18:49:35.044396 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:35.044329    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-203de8/test-model-203de8-kserve-6d79f8dd8f-zf8b4" podStartSLOduration=2.044307726 podStartE2EDuration="2.044307726s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:49:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:49:35.041434285 +0000 UTC m=+1910.198832789" watchObservedRunningTime="2026-07-06 18:49:35.044307726 +0000 UTC m=+1910.201706214"
Jul 06 18:49:35.345793 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:35.345757    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-203de8-openshift-default-5f878c54b5-8c99l"
Jul 06 18:49:35.346072 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:35.346047    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-203de8-openshift-default-5f878c54b5-8c99l"
Jul 06 18:49:35.346736 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:35.346722    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-203de8-openshift-default-5f878c54b5-8c99l"
Jul 06 18:49:36.940953 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:36.940920    2580 patch_prober.go:28] interesting pod/e2e-inf-b-aea86e-openshift-default-6ff5849847-fg5tb container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body=
Jul 06 18:49:36.941437 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:36.940972    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-aea86e-openshift-default-6ff5849847-fg5tb" podUID="2099861f-1605-478e-99ac-224ce565f19c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused"
Jul 06 18:49:37.036436 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:37.036404    2580 generic.go:356] "Generic (PLEG): container finished" podID="2099861f-1605-478e-99ac-224ce565f19c" containerID="0f85ee3e077f4988135316f02809516ec171132fac169687f158781fb58b3e13" exitCode=0
Jul 06 18:49:37.036610 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:37.036482    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-aea86e-openshift-default-6ff5849847-fg5tb" event={"ID":"2099861f-1605-478e-99ac-224ce565f19c","Type":"ContainerDied","Data":"0f85ee3e077f4988135316f02809516ec171132fac169687f158781fb58b3e13"}
Jul 06 18:49:37.036939 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:37.036924    2580 scope.go:117] "RemoveContainer" containerID="0f85ee3e077f4988135316f02809516ec171132fac169687f158781fb58b3e13"
Jul 06 18:49:37.037966 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:37.037937    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 06 18:49:37.038054 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:37.038039    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 06 18:49:37.038113 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:37.038085    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 06 18:49:38.046856 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:38.046812    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-aea86e-openshift-default-6ff5849847-fg5tb" event={"ID":"2099861f-1605-478e-99ac-224ce565f19c","Type":"ContainerStarted","Data":"c02c00f70d4767221ff728e0f0aae439e90e7c1f2d02f185c8668d8c51df6672"}
Jul 06 18:49:38.778604 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:38.778568    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-aea86e-openshift-default-6ff5849847-fg5tb"
Jul 06 18:49:38.778881 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:38.778847    2580 patch_prober.go:28] interesting pod/e2e-inf-b-aea86e-openshift-default-6ff5849847-fg5tb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body=
Jul 06 18:49:38.778982 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:38.778949    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-aea86e-openshift-default-6ff5849847-fg5tb" podUID="2099861f-1605-478e-99ac-224ce565f19c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused"
Jul 06 18:49:39.779200 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:39.779161    2580 patch_prober.go:28] interesting pod/e2e-inf-b-aea86e-openshift-default-6ff5849847-fg5tb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body=
Jul 06 18:49:39.779635 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:39.779220    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-aea86e-openshift-default-6ff5849847-fg5tb" podUID="2099861f-1605-478e-99ac-224ce565f19c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused"
Jul 06 18:49:40.779370 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:40.779330    2580 patch_prober.go:28] interesting pod/e2e-inf-b-aea86e-openshift-default-6ff5849847-fg5tb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body=
Jul 06 18:49:40.779783 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:40.779402    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-aea86e-openshift-default-6ff5849847-fg5tb" podUID="2099861f-1605-478e-99ac-224ce565f19c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused"
Jul 06 18:49:41.779298 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:41.779260    2580 patch_prober.go:28] interesting pod/e2e-inf-b-aea86e-openshift-default-6ff5849847-fg5tb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body=
Jul 06 18:49:41.779483 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:41.779318    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-aea86e-openshift-default-6ff5849847-fg5tb" podUID="2099861f-1605-478e-99ac-224ce565f19c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused"
Jul 06 18:49:42.783059 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:42.783000    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-aea86e-openshift-default-6ff5849847-fg5tb"
Jul 06 18:49:42.783495 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:42.783312    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-aea86e-openshift-default-6ff5849847-fg5tb"
Jul 06 18:49:42.783912 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:42.783894    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-aea86e-openshift-default-6ff5849847-fg5tb"
Jul 06 18:49:44.259601 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:44.259564    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-203de8/test-model-203de8-kserve-6d79f8dd8f-zf8b4"
Jul 06 18:49:44.260073 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:44.259618    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-203de8/test-model-203de8-kserve-6d79f8dd8f-zf8b4"
Jul 06 18:49:44.272381 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:44.272355    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-203de8/test-model-203de8-kserve-6d79f8dd8f-zf8b4"
Jul 06 18:49:45.095465 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:49:45.095435    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-203de8/test-model-203de8-kserve-6d79f8dd8f-zf8b4"
Jul 06 18:50:31.377851 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:50:31.377811    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-203de8/test-model-203de8-kserve-6d79f8dd8f-zf8b4"]
Jul 06 18:50:33.397811 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:50:33.397776    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-203de8-5f967d57dd-jlftr"]
Jul 06 18:50:33.398319 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:50:33.398043    2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-inf-a-203de8-5f967d57dd-jlftr" podUID="3e7a445c-2e56-4fbe-8e0c-0e38433174f7" containerName="maas-api" containerID="cri-o://b65b2f9af0c12db13cf15d9eca69a77ea6939200fa67c4388892a966911ad498" gracePeriod=30
Jul 06 18:50:33.655075 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:50:33.654982    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-203de8-5f967d57dd-jlftr"
Jul 06 18:50:33.703104 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:50:33.703074    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l2mcl\" (UniqueName: \"kubernetes.io/projected/3e7a445c-2e56-4fbe-8e0c-0e38433174f7-kube-api-access-l2mcl\") pod \"3e7a445c-2e56-4fbe-8e0c-0e38433174f7\" (UID: \"3e7a445c-2e56-4fbe-8e0c-0e38433174f7\") "
Jul 06 18:50:33.703277 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:50:33.703182    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3e7a445c-2e56-4fbe-8e0c-0e38433174f7-maas-api-tls\") pod \"3e7a445c-2e56-4fbe-8e0c-0e38433174f7\" (UID: \"3e7a445c-2e56-4fbe-8e0c-0e38433174f7\") "
Jul 06 18:50:33.703277 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:50:33.703241    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3e7a445c-2e56-4fbe-8e0c-0e38433174f7-openshift-service-ca\") pod \"3e7a445c-2e56-4fbe-8e0c-0e38433174f7\" (UID: \"3e7a445c-2e56-4fbe-8e0c-0e38433174f7\") "
Jul 06 18:50:33.703711 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:50:33.703672    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3e7a445c-2e56-4fbe-8e0c-0e38433174f7-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "3e7a445c-2e56-4fbe-8e0c-0e38433174f7" (UID: "3e7a445c-2e56-4fbe-8e0c-0e38433174f7"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 18:50:33.705246 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:50:33.705213    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3e7a445c-2e56-4fbe-8e0c-0e38433174f7-kube-api-access-l2mcl" (OuterVolumeSpecName: "kube-api-access-l2mcl") pod "3e7a445c-2e56-4fbe-8e0c-0e38433174f7" (UID: "3e7a445c-2e56-4fbe-8e0c-0e38433174f7"). InnerVolumeSpecName "kube-api-access-l2mcl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 18:50:33.705702 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:50:33.705683    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3e7a445c-2e56-4fbe-8e0c-0e38433174f7-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "3e7a445c-2e56-4fbe-8e0c-0e38433174f7" (UID: "3e7a445c-2e56-4fbe-8e0c-0e38433174f7"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 18:50:33.804461 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:50:33.804410    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3e7a445c-2e56-4fbe-8e0c-0e38433174f7-maas-api-tls\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:50:33.804461 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:50:33.804454    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3e7a445c-2e56-4fbe-8e0c-0e38433174f7-openshift-service-ca\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:50:33.804461 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:50:33.804465    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l2mcl\" (UniqueName: \"kubernetes.io/projected/3e7a445c-2e56-4fbe-8e0c-0e38433174f7-kube-api-access-l2mcl\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:50:34.321944 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:50:34.321907    2580 generic.go:356] "Generic (PLEG): container finished" podID="3e7a445c-2e56-4fbe-8e0c-0e38433174f7" containerID="b65b2f9af0c12db13cf15d9eca69a77ea6939200fa67c4388892a966911ad498" exitCode=0
Jul 06 18:50:34.321944 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:50:34.321946    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-203de8-5f967d57dd-jlftr"
Jul 06 18:50:34.322241 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:50:34.321973    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-203de8-5f967d57dd-jlftr" event={"ID":"3e7a445c-2e56-4fbe-8e0c-0e38433174f7","Type":"ContainerDied","Data":"b65b2f9af0c12db13cf15d9eca69a77ea6939200fa67c4388892a966911ad498"}
Jul 06 18:50:34.322241 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:50:34.322042    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-203de8-5f967d57dd-jlftr" event={"ID":"3e7a445c-2e56-4fbe-8e0c-0e38433174f7","Type":"ContainerDied","Data":"06fdf366cfbf01cfe0afc015ea783057cc89504b722291214126c550a7667229"}
Jul 06 18:50:34.322241 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:50:34.322070    2580 scope.go:117] "RemoveContainer" containerID="b65b2f9af0c12db13cf15d9eca69a77ea6939200fa67c4388892a966911ad498"
Jul 06 18:50:34.331583 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:50:34.331562    2580 scope.go:117] "RemoveContainer" containerID="b65b2f9af0c12db13cf15d9eca69a77ea6939200fa67c4388892a966911ad498"
Jul 06 18:50:34.331855 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:50:34.331833    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b65b2f9af0c12db13cf15d9eca69a77ea6939200fa67c4388892a966911ad498\": container with ID starting with b65b2f9af0c12db13cf15d9eca69a77ea6939200fa67c4388892a966911ad498 not found: ID does not exist" containerID="b65b2f9af0c12db13cf15d9eca69a77ea6939200fa67c4388892a966911ad498"
Jul 06 18:50:34.331925 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:50:34.331868    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b65b2f9af0c12db13cf15d9eca69a77ea6939200fa67c4388892a966911ad498"} err="failed to get container status \"b65b2f9af0c12db13cf15d9eca69a77ea6939200fa67c4388892a966911ad498\": rpc error: code = NotFound desc = could not find container \"b65b2f9af0c12db13cf15d9eca69a77ea6939200fa67c4388892a966911ad498\": container with ID starting with b65b2f9af0c12db13cf15d9eca69a77ea6939200fa67c4388892a966911ad498 not found: ID does not exist"
Jul 06 18:50:34.347387 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:50:34.347358    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-203de8-5f967d57dd-jlftr"]
Jul 06 18:50:34.352767 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:50:34.352741    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-203de8-5f967d57dd-jlftr"]
Jul 06 18:50:35.324970 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:50:35.324927    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3e7a445c-2e56-4fbe-8e0c-0e38433174f7" path="/var/lib/kubelet/pods/3e7a445c-2e56-4fbe-8e0c-0e38433174f7/volumes"
Jul 06 18:50:46.399675 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:50:46.399613    2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-203de8/test-model-203de8-kserve-6d79f8dd8f-zf8b4" podUID="a6a5937a-88d6-486a-8e59-3676ef7966e7" containerName="main" containerID="cri-o://ba2fe5d220c15088615854147c149e4299a59a381a24c7554bd09943c7a71e66" gracePeriod=45
Jul 06 18:50:46.658988 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:50:46.658918    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-203de8/test-model-203de8-kserve-6d79f8dd8f-zf8b4"
Jul 06 18:50:46.729059 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:50:46.729025    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a6a5937a-88d6-486a-8e59-3676ef7966e7-tls-certs\") pod \"a6a5937a-88d6-486a-8e59-3676ef7966e7\" (UID: \"a6a5937a-88d6-486a-8e59-3676ef7966e7\") "
Jul 06 18:50:46.729270 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:50:46.729068    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tjc9t\" (UniqueName: \"kubernetes.io/projected/a6a5937a-88d6-486a-8e59-3676ef7966e7-kube-api-access-tjc9t\") pod \"a6a5937a-88d6-486a-8e59-3676ef7966e7\" (UID: \"a6a5937a-88d6-486a-8e59-3676ef7966e7\") "
Jul 06 18:50:46.729270 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:50:46.729089    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a6a5937a-88d6-486a-8e59-3676ef7966e7-tmp-dir\") pod \"a6a5937a-88d6-486a-8e59-3676ef7966e7\" (UID: \"a6a5937a-88d6-486a-8e59-3676ef7966e7\") "
Jul 06 18:50:46.729270 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:50:46.729160    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a6a5937a-88d6-486a-8e59-3676ef7966e7-model-cache\") pod \"a6a5937a-88d6-486a-8e59-3676ef7966e7\" (UID: \"a6a5937a-88d6-486a-8e59-3676ef7966e7\") "
Jul 06 18:50:46.729447 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:50:46.729337    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a6a5937a-88d6-486a-8e59-3676ef7966e7-dshm\") pod \"a6a5937a-88d6-486a-8e59-3676ef7966e7\" (UID: \"a6a5937a-88d6-486a-8e59-3676ef7966e7\") "
Jul 06 18:50:46.729447 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:50:46.729378    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a6a5937a-88d6-486a-8e59-3676ef7966e7-model-cache" (OuterVolumeSpecName: "model-cache") pod "a6a5937a-88d6-486a-8e59-3676ef7966e7" (UID: "a6a5937a-88d6-486a-8e59-3676ef7966e7"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 18:50:46.729447 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:50:46.729440    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a6a5937a-88d6-486a-8e59-3676ef7966e7-home\") pod \"a6a5937a-88d6-486a-8e59-3676ef7966e7\" (UID: \"a6a5937a-88d6-486a-8e59-3676ef7966e7\") "
Jul 06 18:50:46.729573 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:50:46.729448    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a6a5937a-88d6-486a-8e59-3676ef7966e7-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "a6a5937a-88d6-486a-8e59-3676ef7966e7" (UID: "a6a5937a-88d6-486a-8e59-3676ef7966e7"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 18:50:46.729686 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:50:46.729661    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a6a5937a-88d6-486a-8e59-3676ef7966e7-home" (OuterVolumeSpecName: "home") pod "a6a5937a-88d6-486a-8e59-3676ef7966e7" (UID: "a6a5937a-88d6-486a-8e59-3676ef7966e7"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 18:50:46.729835 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:50:46.729745    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a6a5937a-88d6-486a-8e59-3676ef7966e7-tmp-dir\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:50:46.729835 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:50:46.729766    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a6a5937a-88d6-486a-8e59-3676ef7966e7-model-cache\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:50:46.729835 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:50:46.729780    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a6a5937a-88d6-486a-8e59-3676ef7966e7-home\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:50:46.731282 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:50:46.731253    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a6a5937a-88d6-486a-8e59-3676ef7966e7-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a6a5937a-88d6-486a-8e59-3676ef7966e7" (UID: "a6a5937a-88d6-486a-8e59-3676ef7966e7"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 18:50:46.731391 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:50:46.731281    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a6a5937a-88d6-486a-8e59-3676ef7966e7-dshm" (OuterVolumeSpecName: "dshm") pod "a6a5937a-88d6-486a-8e59-3676ef7966e7" (UID: "a6a5937a-88d6-486a-8e59-3676ef7966e7"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 18:50:46.731391 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:50:46.731371    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a6a5937a-88d6-486a-8e59-3676ef7966e7-kube-api-access-tjc9t" (OuterVolumeSpecName: "kube-api-access-tjc9t") pod "a6a5937a-88d6-486a-8e59-3676ef7966e7" (UID: "a6a5937a-88d6-486a-8e59-3676ef7966e7"). InnerVolumeSpecName "kube-api-access-tjc9t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 18:50:46.830838 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:50:46.830786    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a6a5937a-88d6-486a-8e59-3676ef7966e7-dshm\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:50:46.830838 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:50:46.830833    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a6a5937a-88d6-486a-8e59-3676ef7966e7-tls-certs\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:50:46.830838 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:50:46.830845    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tjc9t\" (UniqueName: \"kubernetes.io/projected/a6a5937a-88d6-486a-8e59-3676ef7966e7-kube-api-access-tjc9t\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:50:47.386409 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:50:47.386373    2580 generic.go:356] "Generic (PLEG): container finished" podID="a6a5937a-88d6-486a-8e59-3676ef7966e7" containerID="ba2fe5d220c15088615854147c149e4299a59a381a24c7554bd09943c7a71e66" exitCode=0
Jul 06 18:50:47.386623 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:50:47.386416    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-203de8/test-model-203de8-kserve-6d79f8dd8f-zf8b4"
Jul 06 18:50:47.386623 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:50:47.386452    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-203de8/test-model-203de8-kserve-6d79f8dd8f-zf8b4" event={"ID":"a6a5937a-88d6-486a-8e59-3676ef7966e7","Type":"ContainerDied","Data":"ba2fe5d220c15088615854147c149e4299a59a381a24c7554bd09943c7a71e66"}
Jul 06 18:50:47.386623 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:50:47.386490    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-203de8/test-model-203de8-kserve-6d79f8dd8f-zf8b4" event={"ID":"a6a5937a-88d6-486a-8e59-3676ef7966e7","Type":"ContainerDied","Data":"0a56a81dbcbe42d6cd182dc773924b8f050d66bcc124fd82c757e39dde95d5dd"}
Jul 06 18:50:47.386623 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:50:47.386511    2580 scope.go:117] "RemoveContainer" containerID="ba2fe5d220c15088615854147c149e4299a59a381a24c7554bd09943c7a71e66"
Jul 06 18:50:47.395291 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:50:47.395269    2580 scope.go:117] "RemoveContainer" containerID="ba2fe5d220c15088615854147c149e4299a59a381a24c7554bd09943c7a71e66"
Jul 06 18:50:47.395581 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:50:47.395562    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ba2fe5d220c15088615854147c149e4299a59a381a24c7554bd09943c7a71e66\": container with ID starting with ba2fe5d220c15088615854147c149e4299a59a381a24c7554bd09943c7a71e66 not found: ID does not exist" containerID="ba2fe5d220c15088615854147c149e4299a59a381a24c7554bd09943c7a71e66"
Jul 06 18:50:47.395652 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:50:47.395587    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ba2fe5d220c15088615854147c149e4299a59a381a24c7554bd09943c7a71e66"} err="failed to get container status \"ba2fe5d220c15088615854147c149e4299a59a381a24c7554bd09943c7a71e66\": rpc error: code = NotFound desc = could not find container \"ba2fe5d220c15088615854147c149e4299a59a381a24c7554bd09943c7a71e66\": container with ID starting with ba2fe5d220c15088615854147c149e4299a59a381a24c7554bd09943c7a71e66 not found: ID does not exist"
Jul 06 18:50:47.404777 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:50:47.404749    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-203de8/test-model-203de8-kserve-6d79f8dd8f-zf8b4"]
Jul 06 18:50:47.407893 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:50:47.407872    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-203de8/test-model-203de8-kserve-6d79f8dd8f-zf8b4"]
Jul 06 18:50:49.323456 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:50:49.323423    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a6a5937a-88d6-486a-8e59-3676ef7966e7" path="/var/lib/kubelet/pods/a6a5937a-88d6-486a-8e59-3676ef7966e7/volumes"
Jul 06 18:51:14.896300 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:14.896268    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-203de8-openshift-default-5f878c54b5-8c99l"]
Jul 06 18:51:14.896888 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:14.896467    2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-203de8-openshift-default-5f878c54b5-8c99l" podUID="38d03a05-5c33-47fe-ad1d-6866511596bf" containerName="istio-proxy" containerID="cri-o://391b5d623ff0ace4dbe037fd398b56b9643d7b169dc253f11f630de201ce1bd9" gracePeriod=30
Jul 06 18:51:15.977498 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:15.977459    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-aea86e-5b9bc9fdc9-bvbn6"]
Jul 06 18:51:15.977969 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:15.977749    2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-inf-b-aea86e-5b9bc9fdc9-bvbn6" podUID="829713c7-f3f2-4f5a-b4b8-44d840743dae" containerName="maas-api" containerID="cri-o://c5f61cd64c2d2b71cca6928d066f7fe1f1dfd0d735498f292592799b80d3cc02" gracePeriod=30
Jul 06 18:51:16.226506 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:16.226482    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-aea86e-5b9bc9fdc9-bvbn6"
Jul 06 18:51:16.299553 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:16.299458    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/829713c7-f3f2-4f5a-b4b8-44d840743dae-openshift-service-ca\") pod \"829713c7-f3f2-4f5a-b4b8-44d840743dae\" (UID: \"829713c7-f3f2-4f5a-b4b8-44d840743dae\") "
Jul 06 18:51:16.299553 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:16.299521    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/829713c7-f3f2-4f5a-b4b8-44d840743dae-maas-api-tls\") pod \"829713c7-f3f2-4f5a-b4b8-44d840743dae\" (UID: \"829713c7-f3f2-4f5a-b4b8-44d840743dae\") "
Jul 06 18:51:16.299750 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:16.299558    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h59gd\" (UniqueName: \"kubernetes.io/projected/829713c7-f3f2-4f5a-b4b8-44d840743dae-kube-api-access-h59gd\") pod \"829713c7-f3f2-4f5a-b4b8-44d840743dae\" (UID: \"829713c7-f3f2-4f5a-b4b8-44d840743dae\") "
Jul 06 18:51:16.299903 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:16.299874    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/829713c7-f3f2-4f5a-b4b8-44d840743dae-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "829713c7-f3f2-4f5a-b4b8-44d840743dae" (UID: "829713c7-f3f2-4f5a-b4b8-44d840743dae"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 18:51:16.301652 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:16.301615    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/829713c7-f3f2-4f5a-b4b8-44d840743dae-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "829713c7-f3f2-4f5a-b4b8-44d840743dae" (UID: "829713c7-f3f2-4f5a-b4b8-44d840743dae"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 18:51:16.301652 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:16.301616    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/829713c7-f3f2-4f5a-b4b8-44d840743dae-kube-api-access-h59gd" (OuterVolumeSpecName: "kube-api-access-h59gd") pod "829713c7-f3f2-4f5a-b4b8-44d840743dae" (UID: "829713c7-f3f2-4f5a-b4b8-44d840743dae"). InnerVolumeSpecName "kube-api-access-h59gd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 18:51:16.400358 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:16.400320    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h59gd\" (UniqueName: \"kubernetes.io/projected/829713c7-f3f2-4f5a-b4b8-44d840743dae-kube-api-access-h59gd\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:51:16.400358 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:16.400354    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/829713c7-f3f2-4f5a-b4b8-44d840743dae-openshift-service-ca\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:51:16.400358 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:16.400364    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/829713c7-f3f2-4f5a-b4b8-44d840743dae-maas-api-tls\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:51:16.528688 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:16.528648    2580 generic.go:356] "Generic (PLEG): container finished" podID="829713c7-f3f2-4f5a-b4b8-44d840743dae" containerID="c5f61cd64c2d2b71cca6928d066f7fe1f1dfd0d735498f292592799b80d3cc02" exitCode=0
Jul 06 18:51:16.528852 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:16.528691    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-aea86e-5b9bc9fdc9-bvbn6"
Jul 06 18:51:16.528852 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:16.528723    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-aea86e-5b9bc9fdc9-bvbn6" event={"ID":"829713c7-f3f2-4f5a-b4b8-44d840743dae","Type":"ContainerDied","Data":"c5f61cd64c2d2b71cca6928d066f7fe1f1dfd0d735498f292592799b80d3cc02"}
Jul 06 18:51:16.528852 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:16.528775    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-aea86e-5b9bc9fdc9-bvbn6" event={"ID":"829713c7-f3f2-4f5a-b4b8-44d840743dae","Type":"ContainerDied","Data":"0d1c577ab8d85f52b96099a0214eeca0b9a35c5efac7ae571634691b2ef3bb56"}
Jul 06 18:51:16.528852 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:16.528805    2580 scope.go:117] "RemoveContainer" containerID="c5f61cd64c2d2b71cca6928d066f7fe1f1dfd0d735498f292592799b80d3cc02"
Jul 06 18:51:16.537753 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:16.537734    2580 scope.go:117] "RemoveContainer" containerID="c5f61cd64c2d2b71cca6928d066f7fe1f1dfd0d735498f292592799b80d3cc02"
Jul 06 18:51:16.538027 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:51:16.537993    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c5f61cd64c2d2b71cca6928d066f7fe1f1dfd0d735498f292592799b80d3cc02\": container with ID starting with c5f61cd64c2d2b71cca6928d066f7fe1f1dfd0d735498f292592799b80d3cc02 not found: ID does not exist" containerID="c5f61cd64c2d2b71cca6928d066f7fe1f1dfd0d735498f292592799b80d3cc02"
Jul 06 18:51:16.538097 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:16.538039    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c5f61cd64c2d2b71cca6928d066f7fe1f1dfd0d735498f292592799b80d3cc02"} err="failed to get container status \"c5f61cd64c2d2b71cca6928d066f7fe1f1dfd0d735498f292592799b80d3cc02\": rpc error: code = NotFound desc = could not find container \"c5f61cd64c2d2b71cca6928d066f7fe1f1dfd0d735498f292592799b80d3cc02\": container with ID starting with c5f61cd64c2d2b71cca6928d066f7fe1f1dfd0d735498f292592799b80d3cc02 not found: ID does not exist"
Jul 06 18:51:16.549517 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:16.549494    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-aea86e-5b9bc9fdc9-bvbn6"]
Jul 06 18:51:16.554104 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:16.554051    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-aea86e-5b9bc9fdc9-bvbn6"]
Jul 06 18:51:17.322927 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:17.322894    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="829713c7-f3f2-4f5a-b4b8-44d840743dae" path="/var/lib/kubelet/pods/829713c7-f3f2-4f5a-b4b8-44d840743dae/volumes"
Jul 06 18:51:20.149605 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:20.149577    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-203de8-openshift-default-5f878c54b5-8c99l"
Jul 06 18:51:20.225234 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:20.225145    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/38d03a05-5c33-47fe-ad1d-6866511596bf-istio-podinfo\") pod \"38d03a05-5c33-47fe-ad1d-6866511596bf\" (UID: \"38d03a05-5c33-47fe-ad1d-6866511596bf\") "
Jul 06 18:51:20.225234 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:20.225206    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/38d03a05-5c33-47fe-ad1d-6866511596bf-workload-socket\") pod \"38d03a05-5c33-47fe-ad1d-6866511596bf\" (UID: \"38d03a05-5c33-47fe-ad1d-6866511596bf\") "
Jul 06 18:51:20.225428 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:20.225273    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/38d03a05-5c33-47fe-ad1d-6866511596bf-istio-data\") pod \"38d03a05-5c33-47fe-ad1d-6866511596bf\" (UID: \"38d03a05-5c33-47fe-ad1d-6866511596bf\") "
Jul 06 18:51:20.225428 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:20.225300    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/38d03a05-5c33-47fe-ad1d-6866511596bf-istiod-ca-cert\") pod \"38d03a05-5c33-47fe-ad1d-6866511596bf\" (UID: \"38d03a05-5c33-47fe-ad1d-6866511596bf\") "
Jul 06 18:51:20.225428 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:20.225328    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/38d03a05-5c33-47fe-ad1d-6866511596bf-workload-certs\") pod \"38d03a05-5c33-47fe-ad1d-6866511596bf\" (UID: \"38d03a05-5c33-47fe-ad1d-6866511596bf\") "
Jul 06 18:51:20.225428 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:20.225352    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/38d03a05-5c33-47fe-ad1d-6866511596bf-istio-token\") pod \"38d03a05-5c33-47fe-ad1d-6866511596bf\" (UID: \"38d03a05-5c33-47fe-ad1d-6866511596bf\") "
Jul 06 18:51:20.225428 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:20.225384    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/38d03a05-5c33-47fe-ad1d-6866511596bf-istio-envoy\") pod \"38d03a05-5c33-47fe-ad1d-6866511596bf\" (UID: \"38d03a05-5c33-47fe-ad1d-6866511596bf\") "
Jul 06 18:51:20.225691 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:20.225563    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/38d03a05-5c33-47fe-ad1d-6866511596bf-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "38d03a05-5c33-47fe-ad1d-6866511596bf" (UID: "38d03a05-5c33-47fe-ad1d-6866511596bf"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 18:51:20.225691 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:20.225611    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/38d03a05-5c33-47fe-ad1d-6866511596bf-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "38d03a05-5c33-47fe-ad1d-6866511596bf" (UID: "38d03a05-5c33-47fe-ad1d-6866511596bf"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 18:51:20.225818 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:20.225733    2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/38d03a05-5c33-47fe-ad1d-6866511596bf-workload-socket\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:51:20.225818 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:20.225749    2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/38d03a05-5c33-47fe-ad1d-6866511596bf-workload-certs\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:51:20.225936 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:20.225911    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/38d03a05-5c33-47fe-ad1d-6866511596bf-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "38d03a05-5c33-47fe-ad1d-6866511596bf" (UID: "38d03a05-5c33-47fe-ad1d-6866511596bf"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 18:51:20.226160 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:20.226137    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/38d03a05-5c33-47fe-ad1d-6866511596bf-istio-data" (OuterVolumeSpecName: "istio-data") pod "38d03a05-5c33-47fe-ad1d-6866511596bf" (UID: "38d03a05-5c33-47fe-ad1d-6866511596bf"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 18:51:20.227356 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:20.227337    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/38d03a05-5c33-47fe-ad1d-6866511596bf-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "38d03a05-5c33-47fe-ad1d-6866511596bf" (UID: "38d03a05-5c33-47fe-ad1d-6866511596bf"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 06 18:51:20.227530 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:20.227510    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/38d03a05-5c33-47fe-ad1d-6866511596bf-istio-token" (OuterVolumeSpecName: "istio-token") pod "38d03a05-5c33-47fe-ad1d-6866511596bf" (UID: "38d03a05-5c33-47fe-ad1d-6866511596bf"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 18:51:20.227575 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:20.227564    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/38d03a05-5c33-47fe-ad1d-6866511596bf-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "38d03a05-5c33-47fe-ad1d-6866511596bf" (UID: "38d03a05-5c33-47fe-ad1d-6866511596bf"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 18:51:20.326438 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:20.326401    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/38d03a05-5c33-47fe-ad1d-6866511596bf-credential-socket\") pod \"38d03a05-5c33-47fe-ad1d-6866511596bf\" (UID: \"38d03a05-5c33-47fe-ad1d-6866511596bf\") "
Jul 06 18:51:20.326438 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:20.326440    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pxgnr\" (UniqueName: \"kubernetes.io/projected/38d03a05-5c33-47fe-ad1d-6866511596bf-kube-api-access-pxgnr\") pod \"38d03a05-5c33-47fe-ad1d-6866511596bf\" (UID: \"38d03a05-5c33-47fe-ad1d-6866511596bf\") "
Jul 06 18:51:20.326685 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:20.326585    2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/38d03a05-5c33-47fe-ad1d-6866511596bf-istio-data\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:51:20.326685 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:20.326599    2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/38d03a05-5c33-47fe-ad1d-6866511596bf-istiod-ca-cert\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:51:20.326685 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:20.326610    2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/38d03a05-5c33-47fe-ad1d-6866511596bf-istio-token\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:51:20.326685 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:20.326618    2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/38d03a05-5c33-47fe-ad1d-6866511596bf-istio-envoy\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:51:20.326685 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:20.326626    2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/38d03a05-5c33-47fe-ad1d-6866511596bf-istio-podinfo\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:51:20.326902 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:20.326699    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/38d03a05-5c33-47fe-ad1d-6866511596bf-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "38d03a05-5c33-47fe-ad1d-6866511596bf" (UID: "38d03a05-5c33-47fe-ad1d-6866511596bf"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 18:51:20.328601 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:20.328578    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/38d03a05-5c33-47fe-ad1d-6866511596bf-kube-api-access-pxgnr" (OuterVolumeSpecName: "kube-api-access-pxgnr") pod "38d03a05-5c33-47fe-ad1d-6866511596bf" (UID: "38d03a05-5c33-47fe-ad1d-6866511596bf"). InnerVolumeSpecName "kube-api-access-pxgnr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 18:51:20.427388 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:20.427352    2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/38d03a05-5c33-47fe-ad1d-6866511596bf-credential-socket\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:51:20.427388 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:20.427378    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pxgnr\" (UniqueName: \"kubernetes.io/projected/38d03a05-5c33-47fe-ad1d-6866511596bf-kube-api-access-pxgnr\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:51:20.551404 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:20.551318    2580 generic.go:356] "Generic (PLEG): container finished" podID="38d03a05-5c33-47fe-ad1d-6866511596bf" containerID="391b5d623ff0ace4dbe037fd398b56b9643d7b169dc253f11f630de201ce1bd9" exitCode=0
Jul 06 18:51:20.551404 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:20.551369    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-203de8-openshift-default-5f878c54b5-8c99l"
Jul 06 18:51:20.551601 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:20.551406    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-203de8-openshift-default-5f878c54b5-8c99l" event={"ID":"38d03a05-5c33-47fe-ad1d-6866511596bf","Type":"ContainerDied","Data":"391b5d623ff0ace4dbe037fd398b56b9643d7b169dc253f11f630de201ce1bd9"}
Jul 06 18:51:20.551601 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:20.551457    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-203de8-openshift-default-5f878c54b5-8c99l" event={"ID":"38d03a05-5c33-47fe-ad1d-6866511596bf","Type":"ContainerDied","Data":"b797d9041e9dc02a1d4666d478b9af0b19fb75fc1719833904d0af5501e1390a"}
Jul 06 18:51:20.551601 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:20.551480    2580 scope.go:117] "RemoveContainer" containerID="391b5d623ff0ace4dbe037fd398b56b9643d7b169dc253f11f630de201ce1bd9"
Jul 06 18:51:20.563979 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:20.561983    2580 scope.go:117] "RemoveContainer" containerID="34aee067e703743f847d92657b9b8864ca074f3e6977762884aa15ba90a7681b"
Jul 06 18:51:20.570693 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:20.570675    2580 scope.go:117] "RemoveContainer" containerID="391b5d623ff0ace4dbe037fd398b56b9643d7b169dc253f11f630de201ce1bd9"
Jul 06 18:51:20.570953 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:51:20.570932    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"391b5d623ff0ace4dbe037fd398b56b9643d7b169dc253f11f630de201ce1bd9\": container with ID starting with 391b5d623ff0ace4dbe037fd398b56b9643d7b169dc253f11f630de201ce1bd9 not found: ID does not exist" containerID="391b5d623ff0ace4dbe037fd398b56b9643d7b169dc253f11f630de201ce1bd9"
Jul 06 18:51:20.570990 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:20.570962    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"391b5d623ff0ace4dbe037fd398b56b9643d7b169dc253f11f630de201ce1bd9"} err="failed to get container status \"391b5d623ff0ace4dbe037fd398b56b9643d7b169dc253f11f630de201ce1bd9\": rpc error: code = NotFound desc = could not find container \"391b5d623ff0ace4dbe037fd398b56b9643d7b169dc253f11f630de201ce1bd9\": container with ID starting with 391b5d623ff0ace4dbe037fd398b56b9643d7b169dc253f11f630de201ce1bd9 not found: ID does not exist"
Jul 06 18:51:20.570990 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:20.570978    2580 scope.go:117] "RemoveContainer" containerID="34aee067e703743f847d92657b9b8864ca074f3e6977762884aa15ba90a7681b"
Jul 06 18:51:20.571251 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:51:20.571233    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"34aee067e703743f847d92657b9b8864ca074f3e6977762884aa15ba90a7681b\": container with ID starting with 34aee067e703743f847d92657b9b8864ca074f3e6977762884aa15ba90a7681b not found: ID does not exist" containerID="34aee067e703743f847d92657b9b8864ca074f3e6977762884aa15ba90a7681b"
Jul 06 18:51:20.571298 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:20.571257    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"34aee067e703743f847d92657b9b8864ca074f3e6977762884aa15ba90a7681b"} err="failed to get container status \"34aee067e703743f847d92657b9b8864ca074f3e6977762884aa15ba90a7681b\": rpc error: code = NotFound desc = could not find container \"34aee067e703743f847d92657b9b8864ca074f3e6977762884aa15ba90a7681b\": container with ID starting with 34aee067e703743f847d92657b9b8864ca074f3e6977762884aa15ba90a7681b not found: ID does not exist"
Jul 06 18:51:20.580753 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:20.580729    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-203de8-openshift-default-5f878c54b5-8c99l"]
Jul 06 18:51:20.584300 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:20.584276    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-203de8-openshift-default-5f878c54b5-8c99l"]
Jul 06 18:51:21.323233 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:21.323206    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="38d03a05-5c33-47fe-ad1d-6866511596bf" path="/var/lib/kubelet/pods/38d03a05-5c33-47fe-ad1d-6866511596bf/volumes"
Jul 06 18:51:21.909768 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:21.909735    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-aea86e-openshift-default-6ff5849847-fg5tb"]
Jul 06 18:51:21.909973 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:21.909947    2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-aea86e-openshift-default-6ff5849847-fg5tb" podUID="2099861f-1605-478e-99ac-224ce565f19c" containerName="istio-proxy" containerID="cri-o://c02c00f70d4767221ff728e0f0aae439e90e7c1f2d02f185c8668d8c51df6672" gracePeriod=30
Jul 06 18:51:22.803229 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:22.803194    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-05069e-openshift-default-6d4f7bc849-6xhzb"]
Jul 06 18:51:22.803810 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:22.803790    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3e7a445c-2e56-4fbe-8e0c-0e38433174f7" containerName="maas-api"
Jul 06 18:51:22.803901 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:22.803813    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="3e7a445c-2e56-4fbe-8e0c-0e38433174f7" containerName="maas-api"
Jul 06 18:51:22.803901 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:22.803826    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="38d03a05-5c33-47fe-ad1d-6866511596bf" containerName="istio-proxy"
Jul 06 18:51:22.803901 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:22.803835    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="38d03a05-5c33-47fe-ad1d-6866511596bf" containerName="istio-proxy"
Jul 06 18:51:22.803901 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:22.803848    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="38d03a05-5c33-47fe-ad1d-6866511596bf" containerName="istio-proxy"
Jul 06 18:51:22.803901 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:22.803857    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="38d03a05-5c33-47fe-ad1d-6866511596bf" containerName="istio-proxy"
Jul 06 18:51:22.803901 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:22.803875    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="829713c7-f3f2-4f5a-b4b8-44d840743dae" containerName="maas-api"
Jul 06 18:51:22.803901 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:22.803884    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="829713c7-f3f2-4f5a-b4b8-44d840743dae" containerName="maas-api"
Jul 06 18:51:22.803901 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:22.803894    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a6a5937a-88d6-486a-8e59-3676ef7966e7" containerName="main"
Jul 06 18:51:22.803901 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:22.803902    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="a6a5937a-88d6-486a-8e59-3676ef7966e7" containerName="main"
Jul 06 18:51:22.804360 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:22.804005    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="38d03a05-5c33-47fe-ad1d-6866511596bf" containerName="istio-proxy"
Jul 06 18:51:22.804360 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:22.804037    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="a6a5937a-88d6-486a-8e59-3676ef7966e7" containerName="main"
Jul 06 18:51:22.804360 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:22.804049    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="3e7a445c-2e56-4fbe-8e0c-0e38433174f7" containerName="maas-api"
Jul 06 18:51:22.804360 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:22.804060    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="829713c7-f3f2-4f5a-b4b8-44d840743dae" containerName="maas-api"
Jul 06 18:51:22.808639 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:22.808618    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-05069e-openshift-default-6d4f7bc849-6xhzb"
Jul 06 18:51:22.811403 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:22.811383    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-05069e-openshift-default-dockercfg-xlsw6\""
Jul 06 18:51:22.818027 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:22.817989    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-05069e-openshift-default-6d4f7bc849-6xhzb"]
Jul 06 18:51:22.844938 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:22.844907    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca751fc6-5463-4131-a1ae-d280da8530c7-credential-socket\") pod \"e2e-api-a-05069e-openshift-default-6d4f7bc849-6xhzb\" (UID: \"ca751fc6-5463-4131-a1ae-d280da8530c7\") " pod="openshift-ingress/e2e-api-a-05069e-openshift-default-6d4f7bc849-6xhzb"
Jul 06 18:51:22.845120 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:22.844944    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ca751fc6-5463-4131-a1ae-d280da8530c7-workload-certs\") pod \"e2e-api-a-05069e-openshift-default-6d4f7bc849-6xhzb\" (UID: \"ca751fc6-5463-4131-a1ae-d280da8530c7\") " pod="openshift-ingress/e2e-api-a-05069e-openshift-default-6d4f7bc849-6xhzb"
Jul 06 18:51:22.845120 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:22.844964    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ca751fc6-5463-4131-a1ae-d280da8530c7-istiod-ca-cert\") pod \"e2e-api-a-05069e-openshift-default-6d4f7bc849-6xhzb\" (UID: \"ca751fc6-5463-4131-a1ae-d280da8530c7\") " pod="openshift-ingress/e2e-api-a-05069e-openshift-default-6d4f7bc849-6xhzb"
Jul 06 18:51:22.845120 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:22.845053    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mjxqn\" (UniqueName: \"kubernetes.io/projected/ca751fc6-5463-4131-a1ae-d280da8530c7-kube-api-access-mjxqn\") pod \"e2e-api-a-05069e-openshift-default-6d4f7bc849-6xhzb\" (UID: \"ca751fc6-5463-4131-a1ae-d280da8530c7\") " pod="openshift-ingress/e2e-api-a-05069e-openshift-default-6d4f7bc849-6xhzb"
Jul 06 18:51:22.845120 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:22.845095    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ca751fc6-5463-4131-a1ae-d280da8530c7-istio-data\") pod \"e2e-api-a-05069e-openshift-default-6d4f7bc849-6xhzb\" (UID: \"ca751fc6-5463-4131-a1ae-d280da8530c7\") " pod="openshift-ingress/e2e-api-a-05069e-openshift-default-6d4f7bc849-6xhzb"
Jul 06 18:51:22.845120 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:22.845116    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca751fc6-5463-4131-a1ae-d280da8530c7-workload-socket\") pod \"e2e-api-a-05069e-openshift-default-6d4f7bc849-6xhzb\" (UID: \"ca751fc6-5463-4131-a1ae-d280da8530c7\") " pod="openshift-ingress/e2e-api-a-05069e-openshift-default-6d4f7bc849-6xhzb"
Jul 06 18:51:22.845356 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:22.845173    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ca751fc6-5463-4131-a1ae-d280da8530c7-istio-token\") pod \"e2e-api-a-05069e-openshift-default-6d4f7bc849-6xhzb\" (UID: \"ca751fc6-5463-4131-a1ae-d280da8530c7\") " pod="openshift-ingress/e2e-api-a-05069e-openshift-default-6d4f7bc849-6xhzb"
Jul 06 18:51:22.845356 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:22.845202    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ca751fc6-5463-4131-a1ae-d280da8530c7-istio-envoy\") pod \"e2e-api-a-05069e-openshift-default-6d4f7bc849-6xhzb\" (UID: \"ca751fc6-5463-4131-a1ae-d280da8530c7\") " pod="openshift-ingress/e2e-api-a-05069e-openshift-default-6d4f7bc849-6xhzb"
Jul 06 18:51:22.845356 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:22.845232    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ca751fc6-5463-4131-a1ae-d280da8530c7-istio-podinfo\") pod \"e2e-api-a-05069e-openshift-default-6d4f7bc849-6xhzb\" (UID: \"ca751fc6-5463-4131-a1ae-d280da8530c7\") " pod="openshift-ingress/e2e-api-a-05069e-openshift-default-6d4f7bc849-6xhzb"
Jul 06 18:51:22.945632 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:22.945592    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ca751fc6-5463-4131-a1ae-d280da8530c7-workload-certs\") pod \"e2e-api-a-05069e-openshift-default-6d4f7bc849-6xhzb\" (UID: \"ca751fc6-5463-4131-a1ae-d280da8530c7\") " pod="openshift-ingress/e2e-api-a-05069e-openshift-default-6d4f7bc849-6xhzb"
Jul 06 18:51:22.945632 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:22.945631    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ca751fc6-5463-4131-a1ae-d280da8530c7-istiod-ca-cert\") pod \"e2e-api-a-05069e-openshift-default-6d4f7bc849-6xhzb\" (UID: \"ca751fc6-5463-4131-a1ae-d280da8530c7\") " pod="openshift-ingress/e2e-api-a-05069e-openshift-default-6d4f7bc849-6xhzb"
Jul 06 18:51:22.945873 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:22.945680    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mjxqn\" (UniqueName: \"kubernetes.io/projected/ca751fc6-5463-4131-a1ae-d280da8530c7-kube-api-access-mjxqn\") pod \"e2e-api-a-05069e-openshift-default-6d4f7bc849-6xhzb\" (UID: \"ca751fc6-5463-4131-a1ae-d280da8530c7\") " pod="openshift-ingress/e2e-api-a-05069e-openshift-default-6d4f7bc849-6xhzb"
Jul 06 18:51:22.945873 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:22.945710    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ca751fc6-5463-4131-a1ae-d280da8530c7-istio-data\") pod \"e2e-api-a-05069e-openshift-default-6d4f7bc849-6xhzb\" (UID: \"ca751fc6-5463-4131-a1ae-d280da8530c7\") " pod="openshift-ingress/e2e-api-a-05069e-openshift-default-6d4f7bc849-6xhzb"
Jul 06 18:51:22.945873 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:22.945728    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca751fc6-5463-4131-a1ae-d280da8530c7-workload-socket\") pod \"e2e-api-a-05069e-openshift-default-6d4f7bc849-6xhzb\" (UID: \"ca751fc6-5463-4131-a1ae-d280da8530c7\") " pod="openshift-ingress/e2e-api-a-05069e-openshift-default-6d4f7bc849-6xhzb"
Jul 06 18:51:22.945873 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:22.945763    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ca751fc6-5463-4131-a1ae-d280da8530c7-istio-token\") pod \"e2e-api-a-05069e-openshift-default-6d4f7bc849-6xhzb\" (UID: \"ca751fc6-5463-4131-a1ae-d280da8530c7\") " pod="openshift-ingress/e2e-api-a-05069e-openshift-default-6d4f7bc849-6xhzb"
Jul 06 18:51:22.945873 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:22.945789    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ca751fc6-5463-4131-a1ae-d280da8530c7-istio-envoy\") pod \"e2e-api-a-05069e-openshift-default-6d4f7bc849-6xhzb\" (UID: \"ca751fc6-5463-4131-a1ae-d280da8530c7\") " pod="openshift-ingress/e2e-api-a-05069e-openshift-default-6d4f7bc849-6xhzb"
Jul 06 18:51:22.946166 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:22.946102    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ca751fc6-5463-4131-a1ae-d280da8530c7-workload-certs\") pod \"e2e-api-a-05069e-openshift-default-6d4f7bc849-6xhzb\" (UID: \"ca751fc6-5463-4131-a1ae-d280da8530c7\") " pod="openshift-ingress/e2e-api-a-05069e-openshift-default-6d4f7bc849-6xhzb"
Jul 06 18:51:22.946166 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:22.946134    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ca751fc6-5463-4131-a1ae-d280da8530c7-istio-data\") pod \"e2e-api-a-05069e-openshift-default-6d4f7bc849-6xhzb\" (UID: \"ca751fc6-5463-4131-a1ae-d280da8530c7\") " pod="openshift-ingress/e2e-api-a-05069e-openshift-default-6d4f7bc849-6xhzb"
Jul 06 18:51:22.946277 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:22.946225    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca751fc6-5463-4131-a1ae-d280da8530c7-workload-socket\") pod \"e2e-api-a-05069e-openshift-default-6d4f7bc849-6xhzb\" (UID: \"ca751fc6-5463-4131-a1ae-d280da8530c7\") " pod="openshift-ingress/e2e-api-a-05069e-openshift-default-6d4f7bc849-6xhzb"
Jul 06 18:51:22.946277 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:22.946245    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ca751fc6-5463-4131-a1ae-d280da8530c7-istio-podinfo\") pod \"e2e-api-a-05069e-openshift-default-6d4f7bc849-6xhzb\" (UID: \"ca751fc6-5463-4131-a1ae-d280da8530c7\") " pod="openshift-ingress/e2e-api-a-05069e-openshift-default-6d4f7bc849-6xhzb"
Jul 06 18:51:22.946384 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:22.946310    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca751fc6-5463-4131-a1ae-d280da8530c7-credential-socket\") pod \"e2e-api-a-05069e-openshift-default-6d4f7bc849-6xhzb\" (UID: \"ca751fc6-5463-4131-a1ae-d280da8530c7\") " pod="openshift-ingress/e2e-api-a-05069e-openshift-default-6d4f7bc849-6xhzb"
Jul 06 18:51:22.946537 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:22.946514    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ca751fc6-5463-4131-a1ae-d280da8530c7-istiod-ca-cert\") pod \"e2e-api-a-05069e-openshift-default-6d4f7bc849-6xhzb\" (UID: \"ca751fc6-5463-4131-a1ae-d280da8530c7\") " pod="openshift-ingress/e2e-api-a-05069e-openshift-default-6d4f7bc849-6xhzb"
Jul 06 18:51:22.946642 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:22.946606    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca751fc6-5463-4131-a1ae-d280da8530c7-credential-socket\") pod \"e2e-api-a-05069e-openshift-default-6d4f7bc849-6xhzb\" (UID: \"ca751fc6-5463-4131-a1ae-d280da8530c7\") " pod="openshift-ingress/e2e-api-a-05069e-openshift-default-6d4f7bc849-6xhzb"
Jul 06 18:51:22.948112 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:22.948082    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ca751fc6-5463-4131-a1ae-d280da8530c7-istio-envoy\") pod \"e2e-api-a-05069e-openshift-default-6d4f7bc849-6xhzb\" (UID: \"ca751fc6-5463-4131-a1ae-d280da8530c7\") " pod="openshift-ingress/e2e-api-a-05069e-openshift-default-6d4f7bc849-6xhzb"
Jul 06 18:51:22.948466 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:22.948445    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ca751fc6-5463-4131-a1ae-d280da8530c7-istio-podinfo\") pod \"e2e-api-a-05069e-openshift-default-6d4f7bc849-6xhzb\" (UID: \"ca751fc6-5463-4131-a1ae-d280da8530c7\") " pod="openshift-ingress/e2e-api-a-05069e-openshift-default-6d4f7bc849-6xhzb"
Jul 06 18:51:22.954283 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:22.954255    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ca751fc6-5463-4131-a1ae-d280da8530c7-istio-token\") pod \"e2e-api-a-05069e-openshift-default-6d4f7bc849-6xhzb\" (UID: \"ca751fc6-5463-4131-a1ae-d280da8530c7\") " pod="openshift-ingress/e2e-api-a-05069e-openshift-default-6d4f7bc849-6xhzb"
Jul 06 18:51:22.954502 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:22.954483    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mjxqn\" (UniqueName: \"kubernetes.io/projected/ca751fc6-5463-4131-a1ae-d280da8530c7-kube-api-access-mjxqn\") pod \"e2e-api-a-05069e-openshift-default-6d4f7bc849-6xhzb\" (UID: \"ca751fc6-5463-4131-a1ae-d280da8530c7\") " pod="openshift-ingress/e2e-api-a-05069e-openshift-default-6d4f7bc849-6xhzb"
Jul 06 18:51:23.119629 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:23.119592    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-05069e-openshift-default-6d4f7bc849-6xhzb"
Jul 06 18:51:23.261655 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:23.261632    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-05069e-openshift-default-6d4f7bc849-6xhzb"]
Jul 06 18:51:23.263472 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:51:23.263446    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca751fc6_5463_4131_a1ae_d280da8530c7.slice/crio-b8f5d8f62f59b009c7b245039bebe712a9e7a53d3c1073a25bc2ad1f502886b2 WatchSource:0}: Error finding container b8f5d8f62f59b009c7b245039bebe712a9e7a53d3c1073a25bc2ad1f502886b2: Status 404 returned error can't find the container with id b8f5d8f62f59b009c7b245039bebe712a9e7a53d3c1073a25bc2ad1f502886b2
Jul 06 18:51:23.265600 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:23.265569    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 06 18:51:23.265675 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:23.265636    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 06 18:51:23.265718 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:23.265686    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 06 18:51:23.568413 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:23.568325    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-05069e-openshift-default-6d4f7bc849-6xhzb" event={"ID":"ca751fc6-5463-4131-a1ae-d280da8530c7","Type":"ContainerStarted","Data":"c13af12f4f2885e2abf21a74504a8aae8f96f52697c686504a195de31267b638"}
Jul 06 18:51:23.568413 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:23.568378    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-05069e-openshift-default-6d4f7bc849-6xhzb" event={"ID":"ca751fc6-5463-4131-a1ae-d280da8530c7","Type":"ContainerStarted","Data":"b8f5d8f62f59b009c7b245039bebe712a9e7a53d3c1073a25bc2ad1f502886b2"}
Jul 06 18:51:23.589690 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:23.589627    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-05069e-openshift-default-6d4f7bc849-6xhzb" podStartSLOduration=1.5896127020000002 podStartE2EDuration="1.589612702s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:51:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:51:23.587797063 +0000 UTC m=+2018.745195572" watchObservedRunningTime="2026-07-06 18:51:23.589612702 +0000 UTC m=+2018.747011238"
Jul 06 18:51:24.120192 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:24.120150    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-05069e-openshift-default-6d4f7bc849-6xhzb"
Jul 06 18:51:24.125292 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:24.125263    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-05069e-openshift-default-6d4f7bc849-6xhzb"
Jul 06 18:51:24.578289 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:24.578253    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-05069e-openshift-default-6d4f7bc849-6xhzb"
Jul 06 18:51:24.579363 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:24.579340    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-05069e-openshift-default-6d4f7bc849-6xhzb"
Jul 06 18:51:25.545823 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:25.545789    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-api-a-05069e-858656554-gxpfl"]
Jul 06 18:51:25.546275 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:25.546261    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="38d03a05-5c33-47fe-ad1d-6866511596bf" containerName="istio-proxy"
Jul 06 18:51:25.549538 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:25.549513    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-05069e-858656554-gxpfl"
Jul 06 18:51:25.552350 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:25.552326    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-api-a-05069e-serving-cert\""
Jul 06 18:51:25.562409 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:25.562389    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-a-05069e-858656554-gxpfl"]
Jul 06 18:51:25.567810 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:25.567790    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4vzqg\" (UniqueName: \"kubernetes.io/projected/27afcd27-27f6-4bf3-a1c1-f03943fcdf2c-kube-api-access-4vzqg\") pod \"maas-api-e2e-api-a-05069e-858656554-gxpfl\" (UID: \"27afcd27-27f6-4bf3-a1c1-f03943fcdf2c\") " pod="opendatahub/maas-api-e2e-api-a-05069e-858656554-gxpfl"
Jul 06 18:51:25.567925 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:25.567818    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/27afcd27-27f6-4bf3-a1c1-f03943fcdf2c-maas-api-tls\") pod \"maas-api-e2e-api-a-05069e-858656554-gxpfl\" (UID: \"27afcd27-27f6-4bf3-a1c1-f03943fcdf2c\") " pod="opendatahub/maas-api-e2e-api-a-05069e-858656554-gxpfl"
Jul 06 18:51:25.567925 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:25.567839    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/27afcd27-27f6-4bf3-a1c1-f03943fcdf2c-openshift-service-ca\") pod \"maas-api-e2e-api-a-05069e-858656554-gxpfl\" (UID: \"27afcd27-27f6-4bf3-a1c1-f03943fcdf2c\") " pod="opendatahub/maas-api-e2e-api-a-05069e-858656554-gxpfl"
Jul 06 18:51:25.668641 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:25.668603    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4vzqg\" (UniqueName: \"kubernetes.io/projected/27afcd27-27f6-4bf3-a1c1-f03943fcdf2c-kube-api-access-4vzqg\") pod \"maas-api-e2e-api-a-05069e-858656554-gxpfl\" (UID: \"27afcd27-27f6-4bf3-a1c1-f03943fcdf2c\") " pod="opendatahub/maas-api-e2e-api-a-05069e-858656554-gxpfl"
Jul 06 18:51:25.668641 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:25.668648    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/27afcd27-27f6-4bf3-a1c1-f03943fcdf2c-maas-api-tls\") pod \"maas-api-e2e-api-a-05069e-858656554-gxpfl\" (UID: \"27afcd27-27f6-4bf3-a1c1-f03943fcdf2c\") " pod="opendatahub/maas-api-e2e-api-a-05069e-858656554-gxpfl"
Jul 06 18:51:25.668898 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:25.668684    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/27afcd27-27f6-4bf3-a1c1-f03943fcdf2c-openshift-service-ca\") pod \"maas-api-e2e-api-a-05069e-858656554-gxpfl\" (UID: \"27afcd27-27f6-4bf3-a1c1-f03943fcdf2c\") " pod="opendatahub/maas-api-e2e-api-a-05069e-858656554-gxpfl"
Jul 06 18:51:25.669051 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:51:25.669008    2580 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-api-a-05069e-serving-cert: secret "maas-api-e2e-api-a-05069e-serving-cert" not found
Jul 06 18:51:25.669191 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:51:25.669100    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/27afcd27-27f6-4bf3-a1c1-f03943fcdf2c-maas-api-tls podName:27afcd27-27f6-4bf3-a1c1-f03943fcdf2c nodeName:}" failed. No retries permitted until 2026-07-06 18:51:26.16908031 +0000 UTC m=+2021.326478775 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/27afcd27-27f6-4bf3-a1c1-f03943fcdf2c-maas-api-tls") pod "maas-api-e2e-api-a-05069e-858656554-gxpfl" (UID: "27afcd27-27f6-4bf3-a1c1-f03943fcdf2c") : secret "maas-api-e2e-api-a-05069e-serving-cert" not found
Jul 06 18:51:25.669494 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:25.669475    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/27afcd27-27f6-4bf3-a1c1-f03943fcdf2c-openshift-service-ca\") pod \"maas-api-e2e-api-a-05069e-858656554-gxpfl\" (UID: \"27afcd27-27f6-4bf3-a1c1-f03943fcdf2c\") " pod="opendatahub/maas-api-e2e-api-a-05069e-858656554-gxpfl"
Jul 06 18:51:25.680741 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:25.680714    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4vzqg\" (UniqueName: \"kubernetes.io/projected/27afcd27-27f6-4bf3-a1c1-f03943fcdf2c-kube-api-access-4vzqg\") pod \"maas-api-e2e-api-a-05069e-858656554-gxpfl\" (UID: \"27afcd27-27f6-4bf3-a1c1-f03943fcdf2c\") " pod="opendatahub/maas-api-e2e-api-a-05069e-858656554-gxpfl"
Jul 06 18:51:26.172437 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:26.172396    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/27afcd27-27f6-4bf3-a1c1-f03943fcdf2c-maas-api-tls\") pod \"maas-api-e2e-api-a-05069e-858656554-gxpfl\" (UID: \"27afcd27-27f6-4bf3-a1c1-f03943fcdf2c\") " pod="opendatahub/maas-api-e2e-api-a-05069e-858656554-gxpfl"
Jul 06 18:51:26.174953 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:26.174927    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/27afcd27-27f6-4bf3-a1c1-f03943fcdf2c-maas-api-tls\") pod \"maas-api-e2e-api-a-05069e-858656554-gxpfl\" (UID: \"27afcd27-27f6-4bf3-a1c1-f03943fcdf2c\") " pod="opendatahub/maas-api-e2e-api-a-05069e-858656554-gxpfl"
Jul 06 18:51:26.461480 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:26.461375    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-05069e-858656554-gxpfl"
Jul 06 18:51:26.605365 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:26.605343    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-a-05069e-858656554-gxpfl"]
Jul 06 18:51:26.607310 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:51:26.607279    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod27afcd27_27f6_4bf3_a1c1_f03943fcdf2c.slice/crio-ac344beb784854be5cdb6458c80720b240e32cf1e6a823f4906c5f22fe6e7c82 WatchSource:0}: Error finding container ac344beb784854be5cdb6458c80720b240e32cf1e6a823f4906c5f22fe6e7c82: Status 404 returned error can't find the container with id ac344beb784854be5cdb6458c80720b240e32cf1e6a823f4906c5f22fe6e7c82
Jul 06 18:51:27.260374 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:27.260347    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-aea86e-openshift-default-6ff5849847-fg5tb"
Jul 06 18:51:27.282880 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:27.282846    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2099861f-1605-478e-99ac-224ce565f19c-istio-data\") pod \"2099861f-1605-478e-99ac-224ce565f19c\" (UID: \"2099861f-1605-478e-99ac-224ce565f19c\") "
Jul 06 18:51:27.283091 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:27.282921    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2099861f-1605-478e-99ac-224ce565f19c-istio-envoy\") pod \"2099861f-1605-478e-99ac-224ce565f19c\" (UID: \"2099861f-1605-478e-99ac-224ce565f19c\") "
Jul 06 18:51:27.283091 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:27.282948    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2099861f-1605-478e-99ac-224ce565f19c-workload-certs\") pod \"2099861f-1605-478e-99ac-224ce565f19c\" (UID: \"2099861f-1605-478e-99ac-224ce565f19c\") "
Jul 06 18:51:27.283091 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:27.282974    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2099861f-1605-478e-99ac-224ce565f19c-workload-socket\") pod \"2099861f-1605-478e-99ac-224ce565f19c\" (UID: \"2099861f-1605-478e-99ac-224ce565f19c\") "
Jul 06 18:51:27.283091 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:27.283030    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2099861f-1605-478e-99ac-224ce565f19c-istio-podinfo\") pod \"2099861f-1605-478e-99ac-224ce565f19c\" (UID: \"2099861f-1605-478e-99ac-224ce565f19c\") "
Jul 06 18:51:27.283091 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:27.283081    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2099861f-1605-478e-99ac-224ce565f19c-istio-token\") pod \"2099861f-1605-478e-99ac-224ce565f19c\" (UID: \"2099861f-1605-478e-99ac-224ce565f19c\") "
Jul 06 18:51:27.283386 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:27.283126    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2099861f-1605-478e-99ac-224ce565f19c-istiod-ca-cert\") pod \"2099861f-1605-478e-99ac-224ce565f19c\" (UID: \"2099861f-1605-478e-99ac-224ce565f19c\") "
Jul 06 18:51:27.283386 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:27.283152    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fdsd5\" (UniqueName: \"kubernetes.io/projected/2099861f-1605-478e-99ac-224ce565f19c-kube-api-access-fdsd5\") pod \"2099861f-1605-478e-99ac-224ce565f19c\" (UID: \"2099861f-1605-478e-99ac-224ce565f19c\") "
Jul 06 18:51:27.283386 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:27.283175    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2099861f-1605-478e-99ac-224ce565f19c-credential-socket\") pod \"2099861f-1605-478e-99ac-224ce565f19c\" (UID: \"2099861f-1605-478e-99ac-224ce565f19c\") "
Jul 06 18:51:27.284037 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:27.283550    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2099861f-1605-478e-99ac-224ce565f19c-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "2099861f-1605-478e-99ac-224ce565f19c" (UID: "2099861f-1605-478e-99ac-224ce565f19c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 18:51:27.284037 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:27.283714    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2099861f-1605-478e-99ac-224ce565f19c-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "2099861f-1605-478e-99ac-224ce565f19c" (UID: "2099861f-1605-478e-99ac-224ce565f19c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 18:51:27.284037 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:27.283772    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2099861f-1605-478e-99ac-224ce565f19c-istio-data" (OuterVolumeSpecName: "istio-data") pod "2099861f-1605-478e-99ac-224ce565f19c" (UID: "2099861f-1605-478e-99ac-224ce565f19c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 18:51:27.284037 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:27.283970    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2099861f-1605-478e-99ac-224ce565f19c-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "2099861f-1605-478e-99ac-224ce565f19c" (UID: "2099861f-1605-478e-99ac-224ce565f19c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 18:51:27.284310 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:27.284158    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2099861f-1605-478e-99ac-224ce565f19c-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "2099861f-1605-478e-99ac-224ce565f19c" (UID: "2099861f-1605-478e-99ac-224ce565f19c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 18:51:27.286513 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:27.286462    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2099861f-1605-478e-99ac-224ce565f19c-kube-api-access-fdsd5" (OuterVolumeSpecName: "kube-api-access-fdsd5") pod "2099861f-1605-478e-99ac-224ce565f19c" (UID: "2099861f-1605-478e-99ac-224ce565f19c"). InnerVolumeSpecName "kube-api-access-fdsd5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 18:51:27.286513 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:27.286479    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2099861f-1605-478e-99ac-224ce565f19c-istio-token" (OuterVolumeSpecName: "istio-token") pod "2099861f-1605-478e-99ac-224ce565f19c" (UID: "2099861f-1605-478e-99ac-224ce565f19c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 18:51:27.286758 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:27.286718    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2099861f-1605-478e-99ac-224ce565f19c-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "2099861f-1605-478e-99ac-224ce565f19c" (UID: "2099861f-1605-478e-99ac-224ce565f19c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 06 18:51:27.287075 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:27.287042    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2099861f-1605-478e-99ac-224ce565f19c-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "2099861f-1605-478e-99ac-224ce565f19c" (UID: "2099861f-1605-478e-99ac-224ce565f19c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 18:51:27.384522 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:27.384476    2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2099861f-1605-478e-99ac-224ce565f19c-istiod-ca-cert\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:51:27.384522 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:27.384519    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fdsd5\" (UniqueName: \"kubernetes.io/projected/2099861f-1605-478e-99ac-224ce565f19c-kube-api-access-fdsd5\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:51:27.384771 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:27.384539    2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2099861f-1605-478e-99ac-224ce565f19c-credential-socket\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:51:27.384771 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:27.384554    2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2099861f-1605-478e-99ac-224ce565f19c-istio-data\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:51:27.384771 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:27.384568    2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2099861f-1605-478e-99ac-224ce565f19c-istio-envoy\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:51:27.384771 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:27.384581    2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2099861f-1605-478e-99ac-224ce565f19c-workload-certs\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:51:27.384771 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:27.384593    2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2099861f-1605-478e-99ac-224ce565f19c-workload-socket\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:51:27.384771 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:27.384605    2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2099861f-1605-478e-99ac-224ce565f19c-istio-podinfo\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:51:27.384771 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:27.384617    2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2099861f-1605-478e-99ac-224ce565f19c-istio-token\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:51:27.595246 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:27.595209    2580 generic.go:356] "Generic (PLEG): container finished" podID="2099861f-1605-478e-99ac-224ce565f19c" containerID="c02c00f70d4767221ff728e0f0aae439e90e7c1f2d02f185c8668d8c51df6672" exitCode=0
Jul 06 18:51:27.595447 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:27.595262    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-aea86e-openshift-default-6ff5849847-fg5tb"
Jul 06 18:51:27.595447 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:27.595298    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-aea86e-openshift-default-6ff5849847-fg5tb" event={"ID":"2099861f-1605-478e-99ac-224ce565f19c","Type":"ContainerDied","Data":"c02c00f70d4767221ff728e0f0aae439e90e7c1f2d02f185c8668d8c51df6672"}
Jul 06 18:51:27.595447 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:27.595346    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-aea86e-openshift-default-6ff5849847-fg5tb" event={"ID":"2099861f-1605-478e-99ac-224ce565f19c","Type":"ContainerDied","Data":"cf830b1db60f73f518ad9cdba112e1e9f15eee51e2be3def3c0934e5cce032fd"}
Jul 06 18:51:27.595447 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:27.595367    2580 scope.go:117] "RemoveContainer" containerID="c02c00f70d4767221ff728e0f0aae439e90e7c1f2d02f185c8668d8c51df6672"
Jul 06 18:51:27.596885 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:27.596863    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-05069e-858656554-gxpfl" event={"ID":"27afcd27-27f6-4bf3-a1c1-f03943fcdf2c","Type":"ContainerStarted","Data":"e06dd7cdc45125325131fd71d08bbde278c7ae048c4a4b09a3cb3789ad02a3ad"}
Jul 06 18:51:27.596964 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:27.596909    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-05069e-858656554-gxpfl" event={"ID":"27afcd27-27f6-4bf3-a1c1-f03943fcdf2c","Type":"ContainerStarted","Data":"ac344beb784854be5cdb6458c80720b240e32cf1e6a823f4906c5f22fe6e7c82"}
Jul 06 18:51:27.596964 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:27.596940    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-api-a-05069e-858656554-gxpfl"
Jul 06 18:51:27.606257 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:27.606233    2580 scope.go:117] "RemoveContainer" containerID="0f85ee3e077f4988135316f02809516ec171132fac169687f158781fb58b3e13"
Jul 06 18:51:27.614506 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:27.614479    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-aea86e-openshift-default-6ff5849847-fg5tb"]
Jul 06 18:51:27.615060 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:27.615036    2580 scope.go:117] "RemoveContainer" containerID="c02c00f70d4767221ff728e0f0aae439e90e7c1f2d02f185c8668d8c51df6672"
Jul 06 18:51:27.615347 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:51:27.615329    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c02c00f70d4767221ff728e0f0aae439e90e7c1f2d02f185c8668d8c51df6672\": container with ID starting with c02c00f70d4767221ff728e0f0aae439e90e7c1f2d02f185c8668d8c51df6672 not found: ID does not exist" containerID="c02c00f70d4767221ff728e0f0aae439e90e7c1f2d02f185c8668d8c51df6672"
Jul 06 18:51:27.615434 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:27.615363    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c02c00f70d4767221ff728e0f0aae439e90e7c1f2d02f185c8668d8c51df6672"} err="failed to get container status \"c02c00f70d4767221ff728e0f0aae439e90e7c1f2d02f185c8668d8c51df6672\": rpc error: code = NotFound desc = could not find container \"c02c00f70d4767221ff728e0f0aae439e90e7c1f2d02f185c8668d8c51df6672\": container with ID starting with c02c00f70d4767221ff728e0f0aae439e90e7c1f2d02f185c8668d8c51df6672 not found: ID does not exist"
Jul 06 18:51:27.615434 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:27.615392    2580 scope.go:117] "RemoveContainer" containerID="0f85ee3e077f4988135316f02809516ec171132fac169687f158781fb58b3e13"
Jul 06 18:51:27.615687 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:51:27.615669    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0f85ee3e077f4988135316f02809516ec171132fac169687f158781fb58b3e13\": container with ID starting with 0f85ee3e077f4988135316f02809516ec171132fac169687f158781fb58b3e13 not found: ID does not exist" containerID="0f85ee3e077f4988135316f02809516ec171132fac169687f158781fb58b3e13"
Jul 06 18:51:27.615756 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:27.615697    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0f85ee3e077f4988135316f02809516ec171132fac169687f158781fb58b3e13"} err="failed to get container status \"0f85ee3e077f4988135316f02809516ec171132fac169687f158781fb58b3e13\": rpc error: code = NotFound desc = could not find container \"0f85ee3e077f4988135316f02809516ec171132fac169687f158781fb58b3e13\": container with ID starting with 0f85ee3e077f4988135316f02809516ec171132fac169687f158781fb58b3e13 not found: ID does not exist"
Jul 06 18:51:27.618077 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:27.618054    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-aea86e-openshift-default-6ff5849847-fg5tb"]
Jul 06 18:51:27.635717 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:27.635664    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-api-a-05069e-858656554-gxpfl" podStartSLOduration=2.072739729 podStartE2EDuration="2.635648241s" totalImagesPullingTime="562.908512ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:51:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:51:27.635106163 +0000 UTC m=+2022.792504647" watchObservedRunningTime="2026-07-06 18:51:27.635648241 +0000 UTC m=+2022.793046727"
Jul 06 18:51:29.323222 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:29.323188    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2099861f-1605-478e-99ac-224ce565f19c" path="/var/lib/kubelet/pods/2099861f-1605-478e-99ac-224ce565f19c/volumes"
Jul 06 18:51:30.848862 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:30.848827    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-13bc82-openshift-default-59754b7cbc-9wnkq"]
Jul 06 18:51:30.849393 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:30.849376    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2099861f-1605-478e-99ac-224ce565f19c" containerName="istio-proxy"
Jul 06 18:51:30.849467 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:30.849396    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="2099861f-1605-478e-99ac-224ce565f19c" containerName="istio-proxy"
Jul 06 18:51:30.849467 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:30.849422    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2099861f-1605-478e-99ac-224ce565f19c" containerName="istio-proxy"
Jul 06 18:51:30.849467 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:30.849430    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="2099861f-1605-478e-99ac-224ce565f19c" containerName="istio-proxy"
Jul 06 18:51:30.849627 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:30.849514    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="2099861f-1605-478e-99ac-224ce565f19c" containerName="istio-proxy"
Jul 06 18:51:30.854426 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:30.854401    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-13bc82-openshift-default-59754b7cbc-9wnkq"
Jul 06 18:51:30.857309 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:30.857286    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-13bc82-openshift-default-dockercfg-fdzf7\""
Jul 06 18:51:30.863457 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:30.863432    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-13bc82-openshift-default-59754b7cbc-9wnkq"]
Jul 06 18:51:30.918733 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:30.918690    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/600be5b1-255c-4e18-8ea4-9ab099e616aa-istio-data\") pod \"e2e-api-b-13bc82-openshift-default-59754b7cbc-9wnkq\" (UID: \"600be5b1-255c-4e18-8ea4-9ab099e616aa\") " pod="openshift-ingress/e2e-api-b-13bc82-openshift-default-59754b7cbc-9wnkq"
Jul 06 18:51:30.918926 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:30.918805    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/600be5b1-255c-4e18-8ea4-9ab099e616aa-credential-socket\") pod \"e2e-api-b-13bc82-openshift-default-59754b7cbc-9wnkq\" (UID: \"600be5b1-255c-4e18-8ea4-9ab099e616aa\") " pod="openshift-ingress/e2e-api-b-13bc82-openshift-default-59754b7cbc-9wnkq"
Jul 06 18:51:30.918926 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:30.918853    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/600be5b1-255c-4e18-8ea4-9ab099e616aa-istio-podinfo\") pod \"e2e-api-b-13bc82-openshift-default-59754b7cbc-9wnkq\" (UID: \"600be5b1-255c-4e18-8ea4-9ab099e616aa\") " pod="openshift-ingress/e2e-api-b-13bc82-openshift-default-59754b7cbc-9wnkq"
Jul 06 18:51:30.918926 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:30.918877    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/600be5b1-255c-4e18-8ea4-9ab099e616aa-istiod-ca-cert\") pod \"e2e-api-b-13bc82-openshift-default-59754b7cbc-9wnkq\" (UID: \"600be5b1-255c-4e18-8ea4-9ab099e616aa\") " pod="openshift-ingress/e2e-api-b-13bc82-openshift-default-59754b7cbc-9wnkq"
Jul 06 18:51:30.919072 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:30.918958    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/600be5b1-255c-4e18-8ea4-9ab099e616aa-istio-envoy\") pod \"e2e-api-b-13bc82-openshift-default-59754b7cbc-9wnkq\" (UID: \"600be5b1-255c-4e18-8ea4-9ab099e616aa\") " pod="openshift-ingress/e2e-api-b-13bc82-openshift-default-59754b7cbc-9wnkq"
Jul 06 18:51:30.919072 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:30.918994    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vcvz6\" (UniqueName: \"kubernetes.io/projected/600be5b1-255c-4e18-8ea4-9ab099e616aa-kube-api-access-vcvz6\") pod \"e2e-api-b-13bc82-openshift-default-59754b7cbc-9wnkq\" (UID: \"600be5b1-255c-4e18-8ea4-9ab099e616aa\") " pod="openshift-ingress/e2e-api-b-13bc82-openshift-default-59754b7cbc-9wnkq"
Jul 06 18:51:30.919072 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:30.919059    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/600be5b1-255c-4e18-8ea4-9ab099e616aa-istio-token\") pod \"e2e-api-b-13bc82-openshift-default-59754b7cbc-9wnkq\" (UID: \"600be5b1-255c-4e18-8ea4-9ab099e616aa\") " pod="openshift-ingress/e2e-api-b-13bc82-openshift-default-59754b7cbc-9wnkq"
Jul 06 18:51:30.919186 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:30.919095    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/600be5b1-255c-4e18-8ea4-9ab099e616aa-workload-socket\") pod \"e2e-api-b-13bc82-openshift-default-59754b7cbc-9wnkq\" (UID: \"600be5b1-255c-4e18-8ea4-9ab099e616aa\") " pod="openshift-ingress/e2e-api-b-13bc82-openshift-default-59754b7cbc-9wnkq"
Jul 06 18:51:30.919186 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:30.919110    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/600be5b1-255c-4e18-8ea4-9ab099e616aa-workload-certs\") pod \"e2e-api-b-13bc82-openshift-default-59754b7cbc-9wnkq\" (UID: \"600be5b1-255c-4e18-8ea4-9ab099e616aa\") " pod="openshift-ingress/e2e-api-b-13bc82-openshift-default-59754b7cbc-9wnkq"
Jul 06 18:51:31.020563 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:31.020525    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/600be5b1-255c-4e18-8ea4-9ab099e616aa-istio-data\") pod \"e2e-api-b-13bc82-openshift-default-59754b7cbc-9wnkq\" (UID: \"600be5b1-255c-4e18-8ea4-9ab099e616aa\") " pod="openshift-ingress/e2e-api-b-13bc82-openshift-default-59754b7cbc-9wnkq"
Jul 06 18:51:31.020737 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:31.020595    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/600be5b1-255c-4e18-8ea4-9ab099e616aa-credential-socket\") pod \"e2e-api-b-13bc82-openshift-default-59754b7cbc-9wnkq\" (UID: \"600be5b1-255c-4e18-8ea4-9ab099e616aa\") " pod="openshift-ingress/e2e-api-b-13bc82-openshift-default-59754b7cbc-9wnkq"
Jul 06 18:51:31.020737 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:31.020613    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/600be5b1-255c-4e18-8ea4-9ab099e616aa-istio-podinfo\") pod \"e2e-api-b-13bc82-openshift-default-59754b7cbc-9wnkq\" (UID: \"600be5b1-255c-4e18-8ea4-9ab099e616aa\") " pod="openshift-ingress/e2e-api-b-13bc82-openshift-default-59754b7cbc-9wnkq"
Jul 06 18:51:31.020737 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:31.020633    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/600be5b1-255c-4e18-8ea4-9ab099e616aa-istiod-ca-cert\") pod \"e2e-api-b-13bc82-openshift-default-59754b7cbc-9wnkq\" (UID: \"600be5b1-255c-4e18-8ea4-9ab099e616aa\") " pod="openshift-ingress/e2e-api-b-13bc82-openshift-default-59754b7cbc-9wnkq"
Jul 06 18:51:31.020737 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:31.020659    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/600be5b1-255c-4e18-8ea4-9ab099e616aa-istio-envoy\") pod \"e2e-api-b-13bc82-openshift-default-59754b7cbc-9wnkq\" (UID: \"600be5b1-255c-4e18-8ea4-9ab099e616aa\") " pod="openshift-ingress/e2e-api-b-13bc82-openshift-default-59754b7cbc-9wnkq"
Jul 06 18:51:31.020737 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:31.020690    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vcvz6\" (UniqueName: \"kubernetes.io/projected/600be5b1-255c-4e18-8ea4-9ab099e616aa-kube-api-access-vcvz6\") pod \"e2e-api-b-13bc82-openshift-default-59754b7cbc-9wnkq\" (UID: \"600be5b1-255c-4e18-8ea4-9ab099e616aa\") " pod="openshift-ingress/e2e-api-b-13bc82-openshift-default-59754b7cbc-9wnkq"
Jul 06 18:51:31.020737 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:31.020723    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/600be5b1-255c-4e18-8ea4-9ab099e616aa-istio-token\") pod \"e2e-api-b-13bc82-openshift-default-59754b7cbc-9wnkq\" (UID: \"600be5b1-255c-4e18-8ea4-9ab099e616aa\") " pod="openshift-ingress/e2e-api-b-13bc82-openshift-default-59754b7cbc-9wnkq"
Jul 06 18:51:31.021129 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:31.020753    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/600be5b1-255c-4e18-8ea4-9ab099e616aa-workload-socket\") pod \"e2e-api-b-13bc82-openshift-default-59754b7cbc-9wnkq\" (UID: \"600be5b1-255c-4e18-8ea4-9ab099e616aa\") " pod="openshift-ingress/e2e-api-b-13bc82-openshift-default-59754b7cbc-9wnkq"
Jul 06 18:51:31.021129 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:31.020775    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/600be5b1-255c-4e18-8ea4-9ab099e616aa-workload-certs\") pod \"e2e-api-b-13bc82-openshift-default-59754b7cbc-9wnkq\" (UID: \"600be5b1-255c-4e18-8ea4-9ab099e616aa\") " pod="openshift-ingress/e2e-api-b-13bc82-openshift-default-59754b7cbc-9wnkq"
Jul 06 18:51:31.021129 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:31.020982    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/600be5b1-255c-4e18-8ea4-9ab099e616aa-istio-data\") pod \"e2e-api-b-13bc82-openshift-default-59754b7cbc-9wnkq\" (UID: \"600be5b1-255c-4e18-8ea4-9ab099e616aa\") " pod="openshift-ingress/e2e-api-b-13bc82-openshift-default-59754b7cbc-9wnkq"
Jul 06 18:51:31.021129 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:31.021043    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/600be5b1-255c-4e18-8ea4-9ab099e616aa-credential-socket\") pod \"e2e-api-b-13bc82-openshift-default-59754b7cbc-9wnkq\" (UID: \"600be5b1-255c-4e18-8ea4-9ab099e616aa\") " pod="openshift-ingress/e2e-api-b-13bc82-openshift-default-59754b7cbc-9wnkq"
Jul 06 18:51:31.021340 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:31.021131    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/600be5b1-255c-4e18-8ea4-9ab099e616aa-workload-certs\") pod \"e2e-api-b-13bc82-openshift-default-59754b7cbc-9wnkq\" (UID: \"600be5b1-255c-4e18-8ea4-9ab099e616aa\") " pod="openshift-ingress/e2e-api-b-13bc82-openshift-default-59754b7cbc-9wnkq"
Jul 06 18:51:31.021340 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:31.021251    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/600be5b1-255c-4e18-8ea4-9ab099e616aa-workload-socket\") pod \"e2e-api-b-13bc82-openshift-default-59754b7cbc-9wnkq\" (UID: \"600be5b1-255c-4e18-8ea4-9ab099e616aa\") " pod="openshift-ingress/e2e-api-b-13bc82-openshift-default-59754b7cbc-9wnkq"
Jul 06 18:51:31.021769 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:31.021751    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/600be5b1-255c-4e18-8ea4-9ab099e616aa-istiod-ca-cert\") pod \"e2e-api-b-13bc82-openshift-default-59754b7cbc-9wnkq\" (UID: \"600be5b1-255c-4e18-8ea4-9ab099e616aa\") " pod="openshift-ingress/e2e-api-b-13bc82-openshift-default-59754b7cbc-9wnkq"
Jul 06 18:51:31.022986 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:31.022965    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/600be5b1-255c-4e18-8ea4-9ab099e616aa-istio-envoy\") pod \"e2e-api-b-13bc82-openshift-default-59754b7cbc-9wnkq\" (UID: \"600be5b1-255c-4e18-8ea4-9ab099e616aa\") " pod="openshift-ingress/e2e-api-b-13bc82-openshift-default-59754b7cbc-9wnkq"
Jul 06 18:51:31.023256 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:31.023235    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/600be5b1-255c-4e18-8ea4-9ab099e616aa-istio-podinfo\") pod \"e2e-api-b-13bc82-openshift-default-59754b7cbc-9wnkq\" (UID: \"600be5b1-255c-4e18-8ea4-9ab099e616aa\") " pod="openshift-ingress/e2e-api-b-13bc82-openshift-default-59754b7cbc-9wnkq"
Jul 06 18:51:31.029297 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:31.029268    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/600be5b1-255c-4e18-8ea4-9ab099e616aa-istio-token\") pod \"e2e-api-b-13bc82-openshift-default-59754b7cbc-9wnkq\" (UID: \"600be5b1-255c-4e18-8ea4-9ab099e616aa\") " pod="openshift-ingress/e2e-api-b-13bc82-openshift-default-59754b7cbc-9wnkq"
Jul 06 18:51:31.029385 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:31.029295    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vcvz6\" (UniqueName: \"kubernetes.io/projected/600be5b1-255c-4e18-8ea4-9ab099e616aa-kube-api-access-vcvz6\") pod \"e2e-api-b-13bc82-openshift-default-59754b7cbc-9wnkq\" (UID: \"600be5b1-255c-4e18-8ea4-9ab099e616aa\") " pod="openshift-ingress/e2e-api-b-13bc82-openshift-default-59754b7cbc-9wnkq"
Jul 06 18:51:31.167308 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:31.167215    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-13bc82-openshift-default-59754b7cbc-9wnkq"
Jul 06 18:51:31.304808 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:31.304777    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-13bc82-openshift-default-59754b7cbc-9wnkq"]
Jul 06 18:51:31.307271 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:51:31.307235    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod600be5b1_255c_4e18_8ea4_9ab099e616aa.slice/crio-c1e4e3004ee2aa003a4caac0637c1c0dbecd07e69d159122e14f13cfae6dba84 WatchSource:0}: Error finding container c1e4e3004ee2aa003a4caac0637c1c0dbecd07e69d159122e14f13cfae6dba84: Status 404 returned error can't find the container with id c1e4e3004ee2aa003a4caac0637c1c0dbecd07e69d159122e14f13cfae6dba84
Jul 06 18:51:31.309392 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:31.309361    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 06 18:51:31.309463 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:31.309432    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 06 18:51:31.309502 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:31.309465    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 06 18:51:31.618931 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:31.618899    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-13bc82-openshift-default-59754b7cbc-9wnkq" event={"ID":"600be5b1-255c-4e18-8ea4-9ab099e616aa","Type":"ContainerStarted","Data":"41368300b8350c6c6dc5c2e3b41b04e04e5df448d4970003182416974ad358f4"}
Jul 06 18:51:31.619183 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:31.618961    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-13bc82-openshift-default-59754b7cbc-9wnkq" event={"ID":"600be5b1-255c-4e18-8ea4-9ab099e616aa","Type":"ContainerStarted","Data":"c1e4e3004ee2aa003a4caac0637c1c0dbecd07e69d159122e14f13cfae6dba84"}
Jul 06 18:51:31.641210 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:31.641151    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-13bc82-openshift-default-59754b7cbc-9wnkq" podStartSLOduration=1.641135948 podStartE2EDuration="1.641135948s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:51:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:51:31.639295724 +0000 UTC m=+2026.796694235" watchObservedRunningTime="2026-07-06 18:51:31.641135948 +0000 UTC m=+2026.798534433"
Jul 06 18:51:32.167866 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:32.167828    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-13bc82-openshift-default-59754b7cbc-9wnkq"
Jul 06 18:51:32.172933 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:32.172906    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-13bc82-openshift-default-59754b7cbc-9wnkq"
Jul 06 18:51:32.623403 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:32.623373    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-13bc82-openshift-default-59754b7cbc-9wnkq"
Jul 06 18:51:32.624319 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:32.624296    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-13bc82-openshift-default-59754b7cbc-9wnkq"
Jul 06 18:51:33.609357 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:33.609322    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-api-a-05069e-858656554-gxpfl"
Jul 06 18:51:33.812931 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:33.812889    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-api-b-13bc82-bd8446cc-8nms2"]
Jul 06 18:51:33.813405 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:33.813390    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="2099861f-1605-478e-99ac-224ce565f19c" containerName="istio-proxy"
Jul 06 18:51:33.816654 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:33.816631    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-13bc82-bd8446cc-8nms2"
Jul 06 18:51:33.819773 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:33.819750    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-api-b-13bc82-serving-cert\""
Jul 06 18:51:33.837041 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:33.836255    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-b-13bc82-bd8446cc-8nms2"]
Jul 06 18:51:33.861963 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:33.861518    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4f536c0c-4456-4a67-9593-c9d52583aaf6-openshift-service-ca\") pod \"maas-api-e2e-api-b-13bc82-bd8446cc-8nms2\" (UID: \"4f536c0c-4456-4a67-9593-c9d52583aaf6\") " pod="opendatahub/maas-api-e2e-api-b-13bc82-bd8446cc-8nms2"
Jul 06 18:51:33.861963 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:33.861600    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4f536c0c-4456-4a67-9593-c9d52583aaf6-maas-api-tls\") pod \"maas-api-e2e-api-b-13bc82-bd8446cc-8nms2\" (UID: \"4f536c0c-4456-4a67-9593-c9d52583aaf6\") " pod="opendatahub/maas-api-e2e-api-b-13bc82-bd8446cc-8nms2"
Jul 06 18:51:33.861963 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:33.861641    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-stlvn\" (UniqueName: \"kubernetes.io/projected/4f536c0c-4456-4a67-9593-c9d52583aaf6-kube-api-access-stlvn\") pod \"maas-api-e2e-api-b-13bc82-bd8446cc-8nms2\" (UID: \"4f536c0c-4456-4a67-9593-c9d52583aaf6\") " pod="opendatahub/maas-api-e2e-api-b-13bc82-bd8446cc-8nms2"
Jul 06 18:51:33.962761 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:33.962714    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4f536c0c-4456-4a67-9593-c9d52583aaf6-openshift-service-ca\") pod \"maas-api-e2e-api-b-13bc82-bd8446cc-8nms2\" (UID: \"4f536c0c-4456-4a67-9593-c9d52583aaf6\") " pod="opendatahub/maas-api-e2e-api-b-13bc82-bd8446cc-8nms2"
Jul 06 18:51:33.962966 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:33.962784    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4f536c0c-4456-4a67-9593-c9d52583aaf6-maas-api-tls\") pod \"maas-api-e2e-api-b-13bc82-bd8446cc-8nms2\" (UID: \"4f536c0c-4456-4a67-9593-c9d52583aaf6\") " pod="opendatahub/maas-api-e2e-api-b-13bc82-bd8446cc-8nms2"
Jul 06 18:51:33.962966 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:33.962819    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-stlvn\" (UniqueName: \"kubernetes.io/projected/4f536c0c-4456-4a67-9593-c9d52583aaf6-kube-api-access-stlvn\") pod \"maas-api-e2e-api-b-13bc82-bd8446cc-8nms2\" (UID: \"4f536c0c-4456-4a67-9593-c9d52583aaf6\") " pod="opendatahub/maas-api-e2e-api-b-13bc82-bd8446cc-8nms2"
Jul 06 18:51:33.962966 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:51:33.962936    2580 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-api-b-13bc82-serving-cert: secret "maas-api-e2e-api-b-13bc82-serving-cert" not found
Jul 06 18:51:33.963193 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:51:33.962998    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f536c0c-4456-4a67-9593-c9d52583aaf6-maas-api-tls podName:4f536c0c-4456-4a67-9593-c9d52583aaf6 nodeName:}" failed. No retries permitted until 2026-07-06 18:51:34.462983064 +0000 UTC m=+2029.620381528 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/4f536c0c-4456-4a67-9593-c9d52583aaf6-maas-api-tls") pod "maas-api-e2e-api-b-13bc82-bd8446cc-8nms2" (UID: "4f536c0c-4456-4a67-9593-c9d52583aaf6") : secret "maas-api-e2e-api-b-13bc82-serving-cert" not found
Jul 06 18:51:33.963446 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:33.963426    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4f536c0c-4456-4a67-9593-c9d52583aaf6-openshift-service-ca\") pod \"maas-api-e2e-api-b-13bc82-bd8446cc-8nms2\" (UID: \"4f536c0c-4456-4a67-9593-c9d52583aaf6\") " pod="opendatahub/maas-api-e2e-api-b-13bc82-bd8446cc-8nms2"
Jul 06 18:51:33.978372 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:33.978348    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-stlvn\" (UniqueName: \"kubernetes.io/projected/4f536c0c-4456-4a67-9593-c9d52583aaf6-kube-api-access-stlvn\") pod \"maas-api-e2e-api-b-13bc82-bd8446cc-8nms2\" (UID: \"4f536c0c-4456-4a67-9593-c9d52583aaf6\") " pod="opendatahub/maas-api-e2e-api-b-13bc82-bd8446cc-8nms2"
Jul 06 18:51:34.468226 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:34.468191    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4f536c0c-4456-4a67-9593-c9d52583aaf6-maas-api-tls\") pod \"maas-api-e2e-api-b-13bc82-bd8446cc-8nms2\" (UID: \"4f536c0c-4456-4a67-9593-c9d52583aaf6\") " pod="opendatahub/maas-api-e2e-api-b-13bc82-bd8446cc-8nms2"
Jul 06 18:51:34.470657 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:34.470637    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4f536c0c-4456-4a67-9593-c9d52583aaf6-maas-api-tls\") pod \"maas-api-e2e-api-b-13bc82-bd8446cc-8nms2\" (UID: \"4f536c0c-4456-4a67-9593-c9d52583aaf6\") " pod="opendatahub/maas-api-e2e-api-b-13bc82-bd8446cc-8nms2"
Jul 06 18:51:34.732123 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:34.732008    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-13bc82-bd8446cc-8nms2"
Jul 06 18:51:34.914531 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:34.914502    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-b-13bc82-bd8446cc-8nms2"]
Jul 06 18:51:34.916478 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:51:34.916454    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f536c0c_4456_4a67_9593_c9d52583aaf6.slice/crio-7897f239be123d26640050cb0d1f4e71b6cd6f3a88336a1ff15ff3ffdc583db7 WatchSource:0}: Error finding container 7897f239be123d26640050cb0d1f4e71b6cd6f3a88336a1ff15ff3ffdc583db7: Status 404 returned error can't find the container with id 7897f239be123d26640050cb0d1f4e71b6cd6f3a88336a1ff15ff3ffdc583db7
Jul 06 18:51:35.649616 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:35.649568    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-13bc82-bd8446cc-8nms2" event={"ID":"4f536c0c-4456-4a67-9593-c9d52583aaf6","Type":"ContainerStarted","Data":"4dff15b2979e2195fd3a81e80643395204895e10f29372a8a53e92018f9087f5"}
Jul 06 18:51:35.649807 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:35.649641    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-13bc82-bd8446cc-8nms2" event={"ID":"4f536c0c-4456-4a67-9593-c9d52583aaf6","Type":"ContainerStarted","Data":"7897f239be123d26640050cb0d1f4e71b6cd6f3a88336a1ff15ff3ffdc583db7"}
Jul 06 18:51:35.650461 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:35.650431    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-api-b-13bc82-bd8446cc-8nms2"
Jul 06 18:51:35.669472 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:35.669426    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-api-b-13bc82-bd8446cc-8nms2" podStartSLOduration=2.308037831 podStartE2EDuration="2.669409278s" totalImagesPullingTime="361.371447ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:51:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:51:35.667314276 +0000 UTC m=+2030.824712762" watchObservedRunningTime="2026-07-06 18:51:35.669409278 +0000 UTC m=+2030.826807763"
Jul 06 18:51:40.321269 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:40.320188    2580 patch_prober.go:28] interesting pod/e2e-api-a-05069e-openshift-default-6d4f7bc849-6xhzb container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": read tcp 10.133.0.2:39102->10.133.0.63:15021: read: connection reset by peer" start-of-body=
Jul 06 18:51:40.321269 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:40.320255    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-05069e-openshift-default-6d4f7bc849-6xhzb" podUID="ca751fc6-5463-4131-a1ae-d280da8530c7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": read tcp 10.133.0.2:39102->10.133.0.63:15021: read: connection reset by peer"
Jul 06 18:51:40.681221 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:40.681186    2580 generic.go:356] "Generic (PLEG): container finished" podID="ca751fc6-5463-4131-a1ae-d280da8530c7" containerID="c13af12f4f2885e2abf21a74504a8aae8f96f52697c686504a195de31267b638" exitCode=0
Jul 06 18:51:40.681399 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:40.681269    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-05069e-openshift-default-6d4f7bc849-6xhzb" event={"ID":"ca751fc6-5463-4131-a1ae-d280da8530c7","Type":"ContainerDied","Data":"c13af12f4f2885e2abf21a74504a8aae8f96f52697c686504a195de31267b638"}
Jul 06 18:51:40.681714 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:40.681696    2580 scope.go:117] "RemoveContainer" containerID="c13af12f4f2885e2abf21a74504a8aae8f96f52697c686504a195de31267b638"
Jul 06 18:51:40.682805 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:40.682776    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 06 18:51:40.682907 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:40.682845    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 06 18:51:40.682907 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:40.682877    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 06 18:51:41.688163 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:41.688126    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-05069e-openshift-default-6d4f7bc849-6xhzb" event={"ID":"ca751fc6-5463-4131-a1ae-d280da8530c7","Type":"ContainerStarted","Data":"b2d4ebfd658e904fa51c6ec0aab222aa4c7f9a8cb53274de4ec9974252038bf2"}
Jul 06 18:51:42.120368 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:42.120333    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-05069e-openshift-default-6d4f7bc849-6xhzb"
Jul 06 18:51:42.120807 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:42.120783    2580 patch_prober.go:28] interesting pod/e2e-api-a-05069e-openshift-default-6d4f7bc849-6xhzb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body=
Jul 06 18:51:42.120930 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:42.120837    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-05069e-openshift-default-6d4f7bc849-6xhzb" podUID="ca751fc6-5463-4131-a1ae-d280da8530c7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused"
Jul 06 18:51:42.675372 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:42.675332    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-api-b-13bc82-bd8446cc-8nms2"
Jul 06 18:51:43.120997 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:43.120958    2580 patch_prober.go:28] interesting pod/e2e-api-a-05069e-openshift-default-6d4f7bc849-6xhzb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body=
Jul 06 18:51:43.121399 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:43.121027    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-05069e-openshift-default-6d4f7bc849-6xhzb" podUID="ca751fc6-5463-4131-a1ae-d280da8530c7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused"
Jul 06 18:51:44.120133 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:44.120096    2580 patch_prober.go:28] interesting pod/e2e-api-a-05069e-openshift-default-6d4f7bc849-6xhzb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body=
Jul 06 18:51:44.120339 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:44.120162    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-05069e-openshift-default-6d4f7bc849-6xhzb" podUID="ca751fc6-5463-4131-a1ae-d280da8530c7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused"
Jul 06 18:51:45.120127 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:45.120093    2580 patch_prober.go:28] interesting pod/e2e-api-a-05069e-openshift-default-6d4f7bc849-6xhzb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body=
Jul 06 18:51:45.120547 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:45.120148    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-05069e-openshift-default-6d4f7bc849-6xhzb" podUID="ca751fc6-5463-4131-a1ae-d280da8530c7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused"
Jul 06 18:51:46.124114 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:46.124077    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-05069e-openshift-default-6d4f7bc849-6xhzb"
Jul 06 18:51:46.124613 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:46.124594    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-05069e-openshift-default-6d4f7bc849-6xhzb"
Jul 06 18:51:46.125473 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:46.125451    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-05069e-openshift-default-6d4f7bc849-6xhzb"
Jul 06 18:51:46.718669 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:46.718629    2580 generic.go:356] "Generic (PLEG): container finished" podID="600be5b1-255c-4e18-8ea4-9ab099e616aa" containerID="41368300b8350c6c6dc5c2e3b41b04e04e5df448d4970003182416974ad358f4" exitCode=0
Jul 06 18:51:46.718854 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:46.718697    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-13bc82-openshift-default-59754b7cbc-9wnkq" event={"ID":"600be5b1-255c-4e18-8ea4-9ab099e616aa","Type":"ContainerDied","Data":"41368300b8350c6c6dc5c2e3b41b04e04e5df448d4970003182416974ad358f4"}
Jul 06 18:51:46.719211 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:46.719196    2580 scope.go:117] "RemoveContainer" containerID="41368300b8350c6c6dc5c2e3b41b04e04e5df448d4970003182416974ad358f4"
Jul 06 18:51:46.720171 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:46.720148    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 06 18:51:46.720240 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:46.720225    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 06 18:51:46.720277 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:46.720260    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 06 18:51:47.091980 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:47.091944    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-a-05069e-858656554-gxpfl"]
Jul 06 18:51:47.092172 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:47.092151    2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-api-a-05069e-858656554-gxpfl" podUID="27afcd27-27f6-4bf3-a1c1-f03943fcdf2c" containerName="maas-api" containerID="cri-o://e06dd7cdc45125325131fd71d08bbde278c7ae048c4a4b09a3cb3789ad02a3ad" gracePeriod=30
Jul 06 18:51:47.356878 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:47.356851    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-05069e-858656554-gxpfl"
Jul 06 18:51:47.509989 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:47.509954    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4vzqg\" (UniqueName: \"kubernetes.io/projected/27afcd27-27f6-4bf3-a1c1-f03943fcdf2c-kube-api-access-4vzqg\") pod \"27afcd27-27f6-4bf3-a1c1-f03943fcdf2c\" (UID: \"27afcd27-27f6-4bf3-a1c1-f03943fcdf2c\") "
Jul 06 18:51:47.510202 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:47.510001    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/27afcd27-27f6-4bf3-a1c1-f03943fcdf2c-maas-api-tls\") pod \"27afcd27-27f6-4bf3-a1c1-f03943fcdf2c\" (UID: \"27afcd27-27f6-4bf3-a1c1-f03943fcdf2c\") "
Jul 06 18:51:47.510202 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:47.510084    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/27afcd27-27f6-4bf3-a1c1-f03943fcdf2c-openshift-service-ca\") pod \"27afcd27-27f6-4bf3-a1c1-f03943fcdf2c\" (UID: \"27afcd27-27f6-4bf3-a1c1-f03943fcdf2c\") "
Jul 06 18:51:47.510452 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:47.510428    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/27afcd27-27f6-4bf3-a1c1-f03943fcdf2c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "27afcd27-27f6-4bf3-a1c1-f03943fcdf2c" (UID: "27afcd27-27f6-4bf3-a1c1-f03943fcdf2c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 18:51:47.512000 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:47.511975    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/27afcd27-27f6-4bf3-a1c1-f03943fcdf2c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "27afcd27-27f6-4bf3-a1c1-f03943fcdf2c" (UID: "27afcd27-27f6-4bf3-a1c1-f03943fcdf2c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 18:51:47.512105 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:47.512049    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/27afcd27-27f6-4bf3-a1c1-f03943fcdf2c-kube-api-access-4vzqg" (OuterVolumeSpecName: "kube-api-access-4vzqg") pod "27afcd27-27f6-4bf3-a1c1-f03943fcdf2c" (UID: "27afcd27-27f6-4bf3-a1c1-f03943fcdf2c"). InnerVolumeSpecName "kube-api-access-4vzqg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 18:51:47.611632 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:47.611518    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4vzqg\" (UniqueName: \"kubernetes.io/projected/27afcd27-27f6-4bf3-a1c1-f03943fcdf2c-kube-api-access-4vzqg\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:51:47.611632 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:47.611565    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/27afcd27-27f6-4bf3-a1c1-f03943fcdf2c-maas-api-tls\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:51:47.611632 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:47.611577    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/27afcd27-27f6-4bf3-a1c1-f03943fcdf2c-openshift-service-ca\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:51:47.624144 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:47.624100    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-13bc82-openshift-default-59754b7cbc-9wnkq"
Jul 06 18:51:47.726061 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:47.726001    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-13bc82-openshift-default-59754b7cbc-9wnkq" event={"ID":"600be5b1-255c-4e18-8ea4-9ab099e616aa","Type":"ContainerStarted","Data":"27ffe1c71e4802e4bcce487a20062c4d605ad9afdbd0e05b039d0e2182e2ed19"}
Jul 06 18:51:47.728205 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:47.728169    2580 generic.go:356] "Generic (PLEG): container finished" podID="27afcd27-27f6-4bf3-a1c1-f03943fcdf2c" containerID="e06dd7cdc45125325131fd71d08bbde278c7ae048c4a4b09a3cb3789ad02a3ad" exitCode=0
Jul 06 18:51:47.732043 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:47.728546    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-05069e-858656554-gxpfl"
Jul 06 18:51:47.732043 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:47.728849    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-05069e-858656554-gxpfl" event={"ID":"27afcd27-27f6-4bf3-a1c1-f03943fcdf2c","Type":"ContainerDied","Data":"e06dd7cdc45125325131fd71d08bbde278c7ae048c4a4b09a3cb3789ad02a3ad"}
Jul 06 18:51:47.732043 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:47.728893    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-05069e-858656554-gxpfl" event={"ID":"27afcd27-27f6-4bf3-a1c1-f03943fcdf2c","Type":"ContainerDied","Data":"ac344beb784854be5cdb6458c80720b240e32cf1e6a823f4906c5f22fe6e7c82"}
Jul 06 18:51:47.732043 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:47.728924    2580 scope.go:117] "RemoveContainer" containerID="e06dd7cdc45125325131fd71d08bbde278c7ae048c4a4b09a3cb3789ad02a3ad"
Jul 06 18:51:47.754313 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:47.754288    2580 scope.go:117] "RemoveContainer" containerID="e06dd7cdc45125325131fd71d08bbde278c7ae048c4a4b09a3cb3789ad02a3ad"
Jul 06 18:51:47.754934 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:51:47.754864    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e06dd7cdc45125325131fd71d08bbde278c7ae048c4a4b09a3cb3789ad02a3ad\": container with ID starting with e06dd7cdc45125325131fd71d08bbde278c7ae048c4a4b09a3cb3789ad02a3ad not found: ID does not exist" containerID="e06dd7cdc45125325131fd71d08bbde278c7ae048c4a4b09a3cb3789ad02a3ad"
Jul 06 18:51:47.754934 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:47.754902    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e06dd7cdc45125325131fd71d08bbde278c7ae048c4a4b09a3cb3789ad02a3ad"} err="failed to get container status \"e06dd7cdc45125325131fd71d08bbde278c7ae048c4a4b09a3cb3789ad02a3ad\": rpc error: code = NotFound desc = could not find container \"e06dd7cdc45125325131fd71d08bbde278c7ae048c4a4b09a3cb3789ad02a3ad\": container with ID starting with e06dd7cdc45125325131fd71d08bbde278c7ae048c4a4b09a3cb3789ad02a3ad not found: ID does not exist"
Jul 06 18:51:47.767983 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:47.767953    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-a-05069e-858656554-gxpfl"]
Jul 06 18:51:47.771690 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:47.771666    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-api-a-05069e-858656554-gxpfl"]
Jul 06 18:51:48.168202 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:48.168158    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-13bc82-openshift-default-59754b7cbc-9wnkq"
Jul 06 18:51:48.169455 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:48.169420    2580 patch_prober.go:28] interesting pod/e2e-api-b-13bc82-openshift-default-59754b7cbc-9wnkq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body=
Jul 06 18:51:48.169608 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:48.169477    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-13bc82-openshift-default-59754b7cbc-9wnkq" podUID="600be5b1-255c-4e18-8ea4-9ab099e616aa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused"
Jul 06 18:51:49.168685 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:49.168644    2580 patch_prober.go:28] interesting pod/e2e-api-b-13bc82-openshift-default-59754b7cbc-9wnkq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body=
Jul 06 18:51:49.169204 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:49.168732    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-13bc82-openshift-default-59754b7cbc-9wnkq" podUID="600be5b1-255c-4e18-8ea4-9ab099e616aa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused"
Jul 06 18:51:49.323860 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:49.323824    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="27afcd27-27f6-4bf3-a1c1-f03943fcdf2c" path="/var/lib/kubelet/pods/27afcd27-27f6-4bf3-a1c1-f03943fcdf2c/volumes"
Jul 06 18:51:50.168474 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:50.168439    2580 patch_prober.go:28] interesting pod/e2e-api-b-13bc82-openshift-default-59754b7cbc-9wnkq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body=
Jul 06 18:51:50.168659 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:50.168508    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-13bc82-openshift-default-59754b7cbc-9wnkq" podUID="600be5b1-255c-4e18-8ea4-9ab099e616aa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused"
Jul 06 18:51:51.168028 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:51.167987    2580 patch_prober.go:28] interesting pod/e2e-api-b-13bc82-openshift-default-59754b7cbc-9wnkq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body=
Jul 06 18:51:51.168411 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:51.168079    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-13bc82-openshift-default-59754b7cbc-9wnkq" podUID="600be5b1-255c-4e18-8ea4-9ab099e616aa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused"
Jul 06 18:51:52.172244 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:52.172216    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-13bc82-openshift-default-59754b7cbc-9wnkq"
Jul 06 18:51:52.172626 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:52.172491    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-13bc82-openshift-default-59754b7cbc-9wnkq"
Jul 06 18:51:52.173092 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:52.173076    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-13bc82-openshift-default-59754b7cbc-9wnkq"
Jul 06 18:51:53.027094 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:53.027058    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-05069e-openshift-default-6d4f7bc849-6xhzb"]
Jul 06 18:51:53.027321 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:53.027293    2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-05069e-openshift-default-6d4f7bc849-6xhzb" podUID="ca751fc6-5463-4131-a1ae-d280da8530c7" containerName="istio-proxy" containerID="cri-o://b2d4ebfd658e904fa51c6ec0aab222aa4c7f9a8cb53274de4ec9974252038bf2" gracePeriod=30
Jul 06 18:51:54.088615 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:54.088578    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-b-13bc82-bd8446cc-8nms2"]
Jul 06 18:51:54.089119 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:54.088823    2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-api-b-13bc82-bd8446cc-8nms2" podUID="4f536c0c-4456-4a67-9593-c9d52583aaf6" containerName="maas-api" containerID="cri-o://4dff15b2979e2195fd3a81e80643395204895e10f29372a8a53e92018f9087f5" gracePeriod=30
Jul 06 18:51:54.344998 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:54.344933    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-13bc82-bd8446cc-8nms2"
Jul 06 18:51:54.484666 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:54.484625    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4f536c0c-4456-4a67-9593-c9d52583aaf6-maas-api-tls\") pod \"4f536c0c-4456-4a67-9593-c9d52583aaf6\" (UID: \"4f536c0c-4456-4a67-9593-c9d52583aaf6\") "
Jul 06 18:51:54.484846 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:54.484703    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4f536c0c-4456-4a67-9593-c9d52583aaf6-openshift-service-ca\") pod \"4f536c0c-4456-4a67-9593-c9d52583aaf6\" (UID: \"4f536c0c-4456-4a67-9593-c9d52583aaf6\") "
Jul 06 18:51:54.484893 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:54.484869    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-stlvn\" (UniqueName: \"kubernetes.io/projected/4f536c0c-4456-4a67-9593-c9d52583aaf6-kube-api-access-stlvn\") pod \"4f536c0c-4456-4a67-9593-c9d52583aaf6\" (UID: \"4f536c0c-4456-4a67-9593-c9d52583aaf6\") "
Jul 06 18:51:54.485033 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:54.484994    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4f536c0c-4456-4a67-9593-c9d52583aaf6-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "4f536c0c-4456-4a67-9593-c9d52583aaf6" (UID: "4f536c0c-4456-4a67-9593-c9d52583aaf6"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 18:51:54.485226 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:54.485197    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4f536c0c-4456-4a67-9593-c9d52583aaf6-openshift-service-ca\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:51:54.486813 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:54.486792    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4f536c0c-4456-4a67-9593-c9d52583aaf6-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "4f536c0c-4456-4a67-9593-c9d52583aaf6" (UID: "4f536c0c-4456-4a67-9593-c9d52583aaf6"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 18:51:54.486882 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:54.486842    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4f536c0c-4456-4a67-9593-c9d52583aaf6-kube-api-access-stlvn" (OuterVolumeSpecName: "kube-api-access-stlvn") pod "4f536c0c-4456-4a67-9593-c9d52583aaf6" (UID: "4f536c0c-4456-4a67-9593-c9d52583aaf6"). InnerVolumeSpecName "kube-api-access-stlvn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 18:51:54.585998 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:54.585952    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-stlvn\" (UniqueName: \"kubernetes.io/projected/4f536c0c-4456-4a67-9593-c9d52583aaf6-kube-api-access-stlvn\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:51:54.585998 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:54.585989    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4f536c0c-4456-4a67-9593-c9d52583aaf6-maas-api-tls\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:51:54.775178 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:54.775085    2580 generic.go:356] "Generic (PLEG): container finished" podID="4f536c0c-4456-4a67-9593-c9d52583aaf6" containerID="4dff15b2979e2195fd3a81e80643395204895e10f29372a8a53e92018f9087f5" exitCode=0
Jul 06 18:51:54.775178 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:54.775146    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-13bc82-bd8446cc-8nms2"
Jul 06 18:51:54.775421 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:54.775175    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-13bc82-bd8446cc-8nms2" event={"ID":"4f536c0c-4456-4a67-9593-c9d52583aaf6","Type":"ContainerDied","Data":"4dff15b2979e2195fd3a81e80643395204895e10f29372a8a53e92018f9087f5"}
Jul 06 18:51:54.775421 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:54.775235    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-13bc82-bd8446cc-8nms2" event={"ID":"4f536c0c-4456-4a67-9593-c9d52583aaf6","Type":"ContainerDied","Data":"7897f239be123d26640050cb0d1f4e71b6cd6f3a88336a1ff15ff3ffdc583db7"}
Jul 06 18:51:54.775421 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:54.775263    2580 scope.go:117] "RemoveContainer" containerID="4dff15b2979e2195fd3a81e80643395204895e10f29372a8a53e92018f9087f5"
Jul 06 18:51:54.787819 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:54.787725    2580 scope.go:117] "RemoveContainer" containerID="4dff15b2979e2195fd3a81e80643395204895e10f29372a8a53e92018f9087f5"
Jul 06 18:51:54.788173 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:51:54.788131    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4dff15b2979e2195fd3a81e80643395204895e10f29372a8a53e92018f9087f5\": container with ID starting with 4dff15b2979e2195fd3a81e80643395204895e10f29372a8a53e92018f9087f5 not found: ID does not exist" containerID="4dff15b2979e2195fd3a81e80643395204895e10f29372a8a53e92018f9087f5"
Jul 06 18:51:54.788289 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:54.788172    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4dff15b2979e2195fd3a81e80643395204895e10f29372a8a53e92018f9087f5"} err="failed to get container status \"4dff15b2979e2195fd3a81e80643395204895e10f29372a8a53e92018f9087f5\": rpc error: code = NotFound desc = could not find container \"4dff15b2979e2195fd3a81e80643395204895e10f29372a8a53e92018f9087f5\": container with ID starting with 4dff15b2979e2195fd3a81e80643395204895e10f29372a8a53e92018f9087f5 not found: ID does not exist"
Jul 06 18:51:54.802111 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:54.802085    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-b-13bc82-bd8446cc-8nms2"]
Jul 06 18:51:54.808180 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:54.808158    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-api-b-13bc82-bd8446cc-8nms2"]
Jul 06 18:51:55.322805 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:55.322771    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4f536c0c-4456-4a67-9593-c9d52583aaf6" path="/var/lib/kubelet/pods/4f536c0c-4456-4a67-9593-c9d52583aaf6/volumes"
Jul 06 18:51:58.297915 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:58.297848    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-05069e-openshift-default-6d4f7bc849-6xhzb"
Jul 06 18:51:58.316413 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:58.316375    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ca751fc6-5463-4131-a1ae-d280da8530c7-istio-data\") pod \"ca751fc6-5463-4131-a1ae-d280da8530c7\" (UID: \"ca751fc6-5463-4131-a1ae-d280da8530c7\") "
Jul 06 18:51:58.316591 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:58.316436    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ca751fc6-5463-4131-a1ae-d280da8530c7-istio-podinfo\") pod \"ca751fc6-5463-4131-a1ae-d280da8530c7\" (UID: \"ca751fc6-5463-4131-a1ae-d280da8530c7\") "
Jul 06 18:51:58.316591 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:58.316464    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ca751fc6-5463-4131-a1ae-d280da8530c7-istio-token\") pod \"ca751fc6-5463-4131-a1ae-d280da8530c7\" (UID: \"ca751fc6-5463-4131-a1ae-d280da8530c7\") "
Jul 06 18:51:58.316591 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:58.316485    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca751fc6-5463-4131-a1ae-d280da8530c7-credential-socket\") pod \"ca751fc6-5463-4131-a1ae-d280da8530c7\" (UID: \"ca751fc6-5463-4131-a1ae-d280da8530c7\") "
Jul 06 18:51:58.316591 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:58.316515    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ca751fc6-5463-4131-a1ae-d280da8530c7-workload-certs\") pod \"ca751fc6-5463-4131-a1ae-d280da8530c7\" (UID: \"ca751fc6-5463-4131-a1ae-d280da8530c7\") "
Jul 06 18:51:58.316591 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:58.316545    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ca751fc6-5463-4131-a1ae-d280da8530c7-istio-envoy\") pod \"ca751fc6-5463-4131-a1ae-d280da8530c7\" (UID: \"ca751fc6-5463-4131-a1ae-d280da8530c7\") "
Jul 06 18:51:58.316591 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:58.316587    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ca751fc6-5463-4131-a1ae-d280da8530c7-istiod-ca-cert\") pod \"ca751fc6-5463-4131-a1ae-d280da8530c7\" (UID: \"ca751fc6-5463-4131-a1ae-d280da8530c7\") "
Jul 06 18:51:58.316887 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:58.316618    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mjxqn\" (UniqueName: \"kubernetes.io/projected/ca751fc6-5463-4131-a1ae-d280da8530c7-kube-api-access-mjxqn\") pod \"ca751fc6-5463-4131-a1ae-d280da8530c7\" (UID: \"ca751fc6-5463-4131-a1ae-d280da8530c7\") "
Jul 06 18:51:58.316887 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:58.316663    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca751fc6-5463-4131-a1ae-d280da8530c7-workload-socket\") pod \"ca751fc6-5463-4131-a1ae-d280da8530c7\" (UID: \"ca751fc6-5463-4131-a1ae-d280da8530c7\") "
Jul 06 18:51:58.316887 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:58.316809    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca751fc6-5463-4131-a1ae-d280da8530c7-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ca751fc6-5463-4131-a1ae-d280da8530c7" (UID: "ca751fc6-5463-4131-a1ae-d280da8530c7"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 18:51:58.317056 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:58.316985    2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ca751fc6-5463-4131-a1ae-d280da8530c7-workload-certs\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:51:58.317056 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:58.317001    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca751fc6-5463-4131-a1ae-d280da8530c7-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ca751fc6-5463-4131-a1ae-d280da8530c7" (UID: "ca751fc6-5463-4131-a1ae-d280da8530c7"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 18:51:58.317336 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:58.317309    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca751fc6-5463-4131-a1ae-d280da8530c7-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ca751fc6-5463-4131-a1ae-d280da8530c7" (UID: "ca751fc6-5463-4131-a1ae-d280da8530c7"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 18:51:58.317447 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:58.317343    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ca751fc6-5463-4131-a1ae-d280da8530c7-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ca751fc6-5463-4131-a1ae-d280da8530c7" (UID: "ca751fc6-5463-4131-a1ae-d280da8530c7"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 18:51:58.317447 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:58.317367    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca751fc6-5463-4131-a1ae-d280da8530c7-istio-data" (OuterVolumeSpecName: "istio-data") pod "ca751fc6-5463-4131-a1ae-d280da8530c7" (UID: "ca751fc6-5463-4131-a1ae-d280da8530c7"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 18:51:58.318879 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:58.318858    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ca751fc6-5463-4131-a1ae-d280da8530c7-istio-token" (OuterVolumeSpecName: "istio-token") pod "ca751fc6-5463-4131-a1ae-d280da8530c7" (UID: "ca751fc6-5463-4131-a1ae-d280da8530c7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 18:51:58.319269 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:58.319248    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ca751fc6-5463-4131-a1ae-d280da8530c7-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ca751fc6-5463-4131-a1ae-d280da8530c7" (UID: "ca751fc6-5463-4131-a1ae-d280da8530c7"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 06 18:51:58.319363 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:58.319346    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ca751fc6-5463-4131-a1ae-d280da8530c7-kube-api-access-mjxqn" (OuterVolumeSpecName: "kube-api-access-mjxqn") pod "ca751fc6-5463-4131-a1ae-d280da8530c7" (UID: "ca751fc6-5463-4131-a1ae-d280da8530c7"). InnerVolumeSpecName "kube-api-access-mjxqn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 18:51:58.319487 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:58.319467    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca751fc6-5463-4131-a1ae-d280da8530c7-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ca751fc6-5463-4131-a1ae-d280da8530c7" (UID: "ca751fc6-5463-4131-a1ae-d280da8530c7"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 18:51:58.417483 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:58.417443    2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ca751fc6-5463-4131-a1ae-d280da8530c7-istio-token\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:51:58.417483 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:58.417478    2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca751fc6-5463-4131-a1ae-d280da8530c7-credential-socket\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:51:58.417483 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:58.417491    2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ca751fc6-5463-4131-a1ae-d280da8530c7-istio-envoy\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:51:58.417715 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:58.417504    2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ca751fc6-5463-4131-a1ae-d280da8530c7-istiod-ca-cert\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:51:58.417715 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:58.417517    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mjxqn\" (UniqueName: \"kubernetes.io/projected/ca751fc6-5463-4131-a1ae-d280da8530c7-kube-api-access-mjxqn\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:51:58.417715 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:58.417533    2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca751fc6-5463-4131-a1ae-d280da8530c7-workload-socket\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:51:58.417715 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:58.417545    2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ca751fc6-5463-4131-a1ae-d280da8530c7-istio-data\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:51:58.417715 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:58.417558    2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ca751fc6-5463-4131-a1ae-d280da8530c7-istio-podinfo\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:51:58.795622 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:58.795587    2580 generic.go:356] "Generic (PLEG): container finished" podID="ca751fc6-5463-4131-a1ae-d280da8530c7" containerID="b2d4ebfd658e904fa51c6ec0aab222aa4c7f9a8cb53274de4ec9974252038bf2" exitCode=0
Jul 06 18:51:58.795843 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:58.795665    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-05069e-openshift-default-6d4f7bc849-6xhzb"
Jul 06 18:51:58.795843 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:58.795681    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-05069e-openshift-default-6d4f7bc849-6xhzb" event={"ID":"ca751fc6-5463-4131-a1ae-d280da8530c7","Type":"ContainerDied","Data":"b2d4ebfd658e904fa51c6ec0aab222aa4c7f9a8cb53274de4ec9974252038bf2"}
Jul 06 18:51:58.795843 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:58.795756    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-05069e-openshift-default-6d4f7bc849-6xhzb" event={"ID":"ca751fc6-5463-4131-a1ae-d280da8530c7","Type":"ContainerDied","Data":"b8f5d8f62f59b009c7b245039bebe712a9e7a53d3c1073a25bc2ad1f502886b2"}
Jul 06 18:51:58.795843 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:58.795780    2580 scope.go:117] "RemoveContainer" containerID="b2d4ebfd658e904fa51c6ec0aab222aa4c7f9a8cb53274de4ec9974252038bf2"
Jul 06 18:51:58.804757 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:58.804737    2580 scope.go:117] "RemoveContainer" containerID="c13af12f4f2885e2abf21a74504a8aae8f96f52697c686504a195de31267b638"
Jul 06 18:51:58.812546 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:58.812528    2580 scope.go:117] "RemoveContainer" containerID="b2d4ebfd658e904fa51c6ec0aab222aa4c7f9a8cb53274de4ec9974252038bf2"
Jul 06 18:51:58.812792 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:51:58.812772    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b2d4ebfd658e904fa51c6ec0aab222aa4c7f9a8cb53274de4ec9974252038bf2\": container with ID starting with b2d4ebfd658e904fa51c6ec0aab222aa4c7f9a8cb53274de4ec9974252038bf2 not found: ID does not exist" containerID="b2d4ebfd658e904fa51c6ec0aab222aa4c7f9a8cb53274de4ec9974252038bf2"
Jul 06 18:51:58.812837 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:58.812801    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b2d4ebfd658e904fa51c6ec0aab222aa4c7f9a8cb53274de4ec9974252038bf2"} err="failed to get container status \"b2d4ebfd658e904fa51c6ec0aab222aa4c7f9a8cb53274de4ec9974252038bf2\": rpc error: code = NotFound desc = could not find container \"b2d4ebfd658e904fa51c6ec0aab222aa4c7f9a8cb53274de4ec9974252038bf2\": container with ID starting with b2d4ebfd658e904fa51c6ec0aab222aa4c7f9a8cb53274de4ec9974252038bf2 not found: ID does not exist"
Jul 06 18:51:58.812837 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:58.812820    2580 scope.go:117] "RemoveContainer" containerID="c13af12f4f2885e2abf21a74504a8aae8f96f52697c686504a195de31267b638"
Jul 06 18:51:58.813053 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:51:58.813038    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c13af12f4f2885e2abf21a74504a8aae8f96f52697c686504a195de31267b638\": container with ID starting with c13af12f4f2885e2abf21a74504a8aae8f96f52697c686504a195de31267b638 not found: ID does not exist" containerID="c13af12f4f2885e2abf21a74504a8aae8f96f52697c686504a195de31267b638"
Jul 06 18:51:58.813104 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:58.813057    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c13af12f4f2885e2abf21a74504a8aae8f96f52697c686504a195de31267b638"} err="failed to get container status \"c13af12f4f2885e2abf21a74504a8aae8f96f52697c686504a195de31267b638\": rpc error: code = NotFound desc = could not find container \"c13af12f4f2885e2abf21a74504a8aae8f96f52697c686504a195de31267b638\": container with ID starting with c13af12f4f2885e2abf21a74504a8aae8f96f52697c686504a195de31267b638 not found: ID does not exist"
Jul 06 18:51:58.818828 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:58.818804    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-05069e-openshift-default-6d4f7bc849-6xhzb"]
Jul 06 18:51:58.822984 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:58.822964    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-05069e-openshift-default-6d4f7bc849-6xhzb"]
Jul 06 18:51:59.324533 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:51:59.324500    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ca751fc6-5463-4131-a1ae-d280da8530c7" path="/var/lib/kubelet/pods/ca751fc6-5463-4131-a1ae-d280da8530c7/volumes"
Jul 06 18:52:00.171525 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:00.171484    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-13bc82-openshift-default-59754b7cbc-9wnkq"]
Jul 06 18:52:00.171785 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:00.171757    2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-13bc82-openshift-default-59754b7cbc-9wnkq" podUID="600be5b1-255c-4e18-8ea4-9ab099e616aa" containerName="istio-proxy" containerID="cri-o://27ffe1c71e4802e4bcce487a20062c4d605ad9afdbd0e05b039d0e2182e2ed19" gracePeriod=30
Jul 06 18:52:01.066885 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:01.066852    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-8c615a-openshift-default-5b86f7bbcf-kkrr8"]
Jul 06 18:52:01.067292 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:01.067250    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ca751fc6-5463-4131-a1ae-d280da8530c7" containerName="istio-proxy"
Jul 06 18:52:01.067292 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:01.067266    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="ca751fc6-5463-4131-a1ae-d280da8530c7" containerName="istio-proxy"
Jul 06 18:52:01.067292 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:01.067277    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="27afcd27-27f6-4bf3-a1c1-f03943fcdf2c" containerName="maas-api"
Jul 06 18:52:01.067292 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:01.067285    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="27afcd27-27f6-4bf3-a1c1-f03943fcdf2c" containerName="maas-api"
Jul 06 18:52:01.067443 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:01.067304    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4f536c0c-4456-4a67-9593-c9d52583aaf6" containerName="maas-api"
Jul 06 18:52:01.067443 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:01.067310    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="4f536c0c-4456-4a67-9593-c9d52583aaf6" containerName="maas-api"
Jul 06 18:52:01.067443 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:01.067380    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="ca751fc6-5463-4131-a1ae-d280da8530c7" containerName="istio-proxy"
Jul 06 18:52:01.067443 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:01.067389    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="ca751fc6-5463-4131-a1ae-d280da8530c7" containerName="istio-proxy"
Jul 06 18:52:01.067443 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:01.067396    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="4f536c0c-4456-4a67-9593-c9d52583aaf6" containerName="maas-api"
Jul 06 18:52:01.067443 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:01.067402    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="27afcd27-27f6-4bf3-a1c1-f03943fcdf2c" containerName="maas-api"
Jul 06 18:52:01.071662 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:01.071638    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-8c615a-openshift-default-5b86f7bbcf-kkrr8"
Jul 06 18:52:01.074388 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:01.074364    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-8c615a-openshift-default-dockercfg-nh2mm\""
Jul 06 18:52:01.082000 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:01.081978    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-8c615a-openshift-default-5b86f7bbcf-kkrr8"]
Jul 06 18:52:01.141054 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:01.140996    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fef133e7-c5e3-4fa3-a6a5-f57239b3b56e-workload-socket\") pod \"e2e-shared-a-8c615a-openshift-default-5b86f7bbcf-kkrr8\" (UID: \"fef133e7-c5e3-4fa3-a6a5-f57239b3b56e\") " pod="openshift-ingress/e2e-shared-a-8c615a-openshift-default-5b86f7bbcf-kkrr8"
Jul 06 18:52:01.141054 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:01.141050    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fef133e7-c5e3-4fa3-a6a5-f57239b3b56e-workload-certs\") pod \"e2e-shared-a-8c615a-openshift-default-5b86f7bbcf-kkrr8\" (UID: \"fef133e7-c5e3-4fa3-a6a5-f57239b3b56e\") " pod="openshift-ingress/e2e-shared-a-8c615a-openshift-default-5b86f7bbcf-kkrr8"
Jul 06 18:52:01.141332 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:01.141129    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fef133e7-c5e3-4fa3-a6a5-f57239b3b56e-istio-token\") pod \"e2e-shared-a-8c615a-openshift-default-5b86f7bbcf-kkrr8\" (UID: \"fef133e7-c5e3-4fa3-a6a5-f57239b3b56e\") " pod="openshift-ingress/e2e-shared-a-8c615a-openshift-default-5b86f7bbcf-kkrr8"
Jul 06 18:52:01.141332 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:01.141162    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fef133e7-c5e3-4fa3-a6a5-f57239b3b56e-credential-socket\") pod \"e2e-shared-a-8c615a-openshift-default-5b86f7bbcf-kkrr8\" (UID: \"fef133e7-c5e3-4fa3-a6a5-f57239b3b56e\") " pod="openshift-ingress/e2e-shared-a-8c615a-openshift-default-5b86f7bbcf-kkrr8"
Jul 06 18:52:01.141332 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:01.141208    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fef133e7-c5e3-4fa3-a6a5-f57239b3b56e-istio-envoy\") pod \"e2e-shared-a-8c615a-openshift-default-5b86f7bbcf-kkrr8\" (UID: \"fef133e7-c5e3-4fa3-a6a5-f57239b3b56e\") " pod="openshift-ingress/e2e-shared-a-8c615a-openshift-default-5b86f7bbcf-kkrr8"
Jul 06 18:52:01.141332 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:01.141238    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kjjhk\" (UniqueName: \"kubernetes.io/projected/fef133e7-c5e3-4fa3-a6a5-f57239b3b56e-kube-api-access-kjjhk\") pod \"e2e-shared-a-8c615a-openshift-default-5b86f7bbcf-kkrr8\" (UID: \"fef133e7-c5e3-4fa3-a6a5-f57239b3b56e\") " pod="openshift-ingress/e2e-shared-a-8c615a-openshift-default-5b86f7bbcf-kkrr8"
Jul 06 18:52:01.141332 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:01.141279    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fef133e7-c5e3-4fa3-a6a5-f57239b3b56e-istiod-ca-cert\") pod \"e2e-shared-a-8c615a-openshift-default-5b86f7bbcf-kkrr8\" (UID: \"fef133e7-c5e3-4fa3-a6a5-f57239b3b56e\") " pod="openshift-ingress/e2e-shared-a-8c615a-openshift-default-5b86f7bbcf-kkrr8"
Jul 06 18:52:01.141332 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:01.141300    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fef133e7-c5e3-4fa3-a6a5-f57239b3b56e-istio-data\") pod \"e2e-shared-a-8c615a-openshift-default-5b86f7bbcf-kkrr8\" (UID: \"fef133e7-c5e3-4fa3-a6a5-f57239b3b56e\") " pod="openshift-ingress/e2e-shared-a-8c615a-openshift-default-5b86f7bbcf-kkrr8"
Jul 06 18:52:01.141646 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:01.141344    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fef133e7-c5e3-4fa3-a6a5-f57239b3b56e-istio-podinfo\") pod \"e2e-shared-a-8c615a-openshift-default-5b86f7bbcf-kkrr8\" (UID: \"fef133e7-c5e3-4fa3-a6a5-f57239b3b56e\") " pod="openshift-ingress/e2e-shared-a-8c615a-openshift-default-5b86f7bbcf-kkrr8"
Jul 06 18:52:01.242501 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:01.242465    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fef133e7-c5e3-4fa3-a6a5-f57239b3b56e-workload-socket\") pod \"e2e-shared-a-8c615a-openshift-default-5b86f7bbcf-kkrr8\" (UID: \"fef133e7-c5e3-4fa3-a6a5-f57239b3b56e\") " pod="openshift-ingress/e2e-shared-a-8c615a-openshift-default-5b86f7bbcf-kkrr8"
Jul 06 18:52:01.242682 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:01.242510    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fef133e7-c5e3-4fa3-a6a5-f57239b3b56e-workload-certs\") pod \"e2e-shared-a-8c615a-openshift-default-5b86f7bbcf-kkrr8\" (UID: \"fef133e7-c5e3-4fa3-a6a5-f57239b3b56e\") " pod="openshift-ingress/e2e-shared-a-8c615a-openshift-default-5b86f7bbcf-kkrr8"
Jul 06 18:52:01.242682 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:01.242563    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fef133e7-c5e3-4fa3-a6a5-f57239b3b56e-istio-token\") pod \"e2e-shared-a-8c615a-openshift-default-5b86f7bbcf-kkrr8\" (UID: \"fef133e7-c5e3-4fa3-a6a5-f57239b3b56e\") " pod="openshift-ingress/e2e-shared-a-8c615a-openshift-default-5b86f7bbcf-kkrr8"
Jul 06 18:52:01.242682 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:01.242592    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fef133e7-c5e3-4fa3-a6a5-f57239b3b56e-credential-socket\") pod \"e2e-shared-a-8c615a-openshift-default-5b86f7bbcf-kkrr8\" (UID: \"fef133e7-c5e3-4fa3-a6a5-f57239b3b56e\") " pod="openshift-ingress/e2e-shared-a-8c615a-openshift-default-5b86f7bbcf-kkrr8"
Jul 06 18:52:01.242682 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:01.242638    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fef133e7-c5e3-4fa3-a6a5-f57239b3b56e-istio-envoy\") pod \"e2e-shared-a-8c615a-openshift-default-5b86f7bbcf-kkrr8\" (UID: \"fef133e7-c5e3-4fa3-a6a5-f57239b3b56e\") " pod="openshift-ingress/e2e-shared-a-8c615a-openshift-default-5b86f7bbcf-kkrr8"
Jul 06 18:52:01.242682 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:01.242673    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kjjhk\" (UniqueName: \"kubernetes.io/projected/fef133e7-c5e3-4fa3-a6a5-f57239b3b56e-kube-api-access-kjjhk\") pod \"e2e-shared-a-8c615a-openshift-default-5b86f7bbcf-kkrr8\" (UID: \"fef133e7-c5e3-4fa3-a6a5-f57239b3b56e\") " pod="openshift-ingress/e2e-shared-a-8c615a-openshift-default-5b86f7bbcf-kkrr8"
Jul 06 18:52:01.242954 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:01.242730    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fef133e7-c5e3-4fa3-a6a5-f57239b3b56e-istiod-ca-cert\") pod \"e2e-shared-a-8c615a-openshift-default-5b86f7bbcf-kkrr8\" (UID: \"fef133e7-c5e3-4fa3-a6a5-f57239b3b56e\") " pod="openshift-ingress/e2e-shared-a-8c615a-openshift-default-5b86f7bbcf-kkrr8"
Jul 06 18:52:01.242954 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:01.242762    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fef133e7-c5e3-4fa3-a6a5-f57239b3b56e-istio-data\") pod \"e2e-shared-a-8c615a-openshift-default-5b86f7bbcf-kkrr8\" (UID: \"fef133e7-c5e3-4fa3-a6a5-f57239b3b56e\") " pod="openshift-ingress/e2e-shared-a-8c615a-openshift-default-5b86f7bbcf-kkrr8"
Jul 06 18:52:01.242954 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:01.242800    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fef133e7-c5e3-4fa3-a6a5-f57239b3b56e-istio-podinfo\") pod \"e2e-shared-a-8c615a-openshift-default-5b86f7bbcf-kkrr8\" (UID: \"fef133e7-c5e3-4fa3-a6a5-f57239b3b56e\") " pod="openshift-ingress/e2e-shared-a-8c615a-openshift-default-5b86f7bbcf-kkrr8"
Jul 06 18:52:01.242954 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:01.242914    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fef133e7-c5e3-4fa3-a6a5-f57239b3b56e-workload-socket\") pod \"e2e-shared-a-8c615a-openshift-default-5b86f7bbcf-kkrr8\" (UID: \"fef133e7-c5e3-4fa3-a6a5-f57239b3b56e\") " pod="openshift-ingress/e2e-shared-a-8c615a-openshift-default-5b86f7bbcf-kkrr8"
Jul 06 18:52:01.243242 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:01.243220    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fef133e7-c5e3-4fa3-a6a5-f57239b3b56e-istio-data\") pod \"e2e-shared-a-8c615a-openshift-default-5b86f7bbcf-kkrr8\" (UID: \"fef133e7-c5e3-4fa3-a6a5-f57239b3b56e\") " pod="openshift-ingress/e2e-shared-a-8c615a-openshift-default-5b86f7bbcf-kkrr8"
Jul 06 18:52:01.243472 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:01.243448    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fef133e7-c5e3-4fa3-a6a5-f57239b3b56e-workload-certs\") pod \"e2e-shared-a-8c615a-openshift-default-5b86f7bbcf-kkrr8\" (UID: \"fef133e7-c5e3-4fa3-a6a5-f57239b3b56e\") " pod="openshift-ingress/e2e-shared-a-8c615a-openshift-default-5b86f7bbcf-kkrr8"
Jul 06 18:52:01.243605 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:01.243536    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fef133e7-c5e3-4fa3-a6a5-f57239b3b56e-istiod-ca-cert\") pod \"e2e-shared-a-8c615a-openshift-default-5b86f7bbcf-kkrr8\" (UID: \"fef133e7-c5e3-4fa3-a6a5-f57239b3b56e\") " pod="openshift-ingress/e2e-shared-a-8c615a-openshift-default-5b86f7bbcf-kkrr8"
Jul 06 18:52:01.243605 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:01.243540    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fef133e7-c5e3-4fa3-a6a5-f57239b3b56e-credential-socket\") pod \"e2e-shared-a-8c615a-openshift-default-5b86f7bbcf-kkrr8\" (UID: \"fef133e7-c5e3-4fa3-a6a5-f57239b3b56e\") " pod="openshift-ingress/e2e-shared-a-8c615a-openshift-default-5b86f7bbcf-kkrr8"
Jul 06 18:52:01.244911 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:01.244888    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fef133e7-c5e3-4fa3-a6a5-f57239b3b56e-istio-envoy\") pod \"e2e-shared-a-8c615a-openshift-default-5b86f7bbcf-kkrr8\" (UID: \"fef133e7-c5e3-4fa3-a6a5-f57239b3b56e\") " pod="openshift-ingress/e2e-shared-a-8c615a-openshift-default-5b86f7bbcf-kkrr8"
Jul 06 18:52:01.245321 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:01.245300    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fef133e7-c5e3-4fa3-a6a5-f57239b3b56e-istio-podinfo\") pod \"e2e-shared-a-8c615a-openshift-default-5b86f7bbcf-kkrr8\" (UID: \"fef133e7-c5e3-4fa3-a6a5-f57239b3b56e\") " pod="openshift-ingress/e2e-shared-a-8c615a-openshift-default-5b86f7bbcf-kkrr8"
Jul 06 18:52:01.251498 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:01.251472    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fef133e7-c5e3-4fa3-a6a5-f57239b3b56e-istio-token\") pod \"e2e-shared-a-8c615a-openshift-default-5b86f7bbcf-kkrr8\" (UID: \"fef133e7-c5e3-4fa3-a6a5-f57239b3b56e\") " pod="openshift-ingress/e2e-shared-a-8c615a-openshift-default-5b86f7bbcf-kkrr8"
Jul 06 18:52:01.251660 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:01.251640    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kjjhk\" (UniqueName: \"kubernetes.io/projected/fef133e7-c5e3-4fa3-a6a5-f57239b3b56e-kube-api-access-kjjhk\") pod \"e2e-shared-a-8c615a-openshift-default-5b86f7bbcf-kkrr8\" (UID: \"fef133e7-c5e3-4fa3-a6a5-f57239b3b56e\") " pod="openshift-ingress/e2e-shared-a-8c615a-openshift-default-5b86f7bbcf-kkrr8"
Jul 06 18:52:01.383522 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:01.383484    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-8c615a-openshift-default-5b86f7bbcf-kkrr8"
Jul 06 18:52:01.531515 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:01.531488    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-8c615a-openshift-default-5b86f7bbcf-kkrr8"]
Jul 06 18:52:01.533717 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:52:01.533682    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfef133e7_c5e3_4fa3_a6a5_f57239b3b56e.slice/crio-8b9f644ff1c34c41d302c5700bf3658b186da8835634c2a1bd98fbc3080bb2a2 WatchSource:0}: Error finding container 8b9f644ff1c34c41d302c5700bf3658b186da8835634c2a1bd98fbc3080bb2a2: Status 404 returned error can't find the container with id 8b9f644ff1c34c41d302c5700bf3658b186da8835634c2a1bd98fbc3080bb2a2
Jul 06 18:52:01.536034 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:01.535981    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 06 18:52:01.536117 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:01.536059    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 06 18:52:01.536117 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:01.536107    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 06 18:52:01.811809 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:01.811708    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-8c615a-openshift-default-5b86f7bbcf-kkrr8" event={"ID":"fef133e7-c5e3-4fa3-a6a5-f57239b3b56e","Type":"ContainerStarted","Data":"97616b491f19f13001a9a37cfb0cb797e1dd901959eeaea648c4cc903f160052"}
Jul 06 18:52:01.811809 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:01.811765    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-8c615a-openshift-default-5b86f7bbcf-kkrr8" event={"ID":"fef133e7-c5e3-4fa3-a6a5-f57239b3b56e","Type":"ContainerStarted","Data":"8b9f644ff1c34c41d302c5700bf3658b186da8835634c2a1bd98fbc3080bb2a2"}
Jul 06 18:52:01.833086 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:01.833035    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-8c615a-openshift-default-5b86f7bbcf-kkrr8" podStartSLOduration=0.833000984 podStartE2EDuration="833.000984ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:52:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:52:01.83120669 +0000 UTC m=+2056.988605176" watchObservedRunningTime="2026-07-06 18:52:01.833000984 +0000 UTC m=+2056.990399515"
Jul 06 18:52:02.383786 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:02.383665    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-8c615a-openshift-default-5b86f7bbcf-kkrr8"
Jul 06 18:52:02.390458 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:02.390429    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-8c615a-openshift-default-5b86f7bbcf-kkrr8"
Jul 06 18:52:02.816707 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:02.816619    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-8c615a-openshift-default-5b86f7bbcf-kkrr8"
Jul 06 18:52:02.817703 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:02.817686    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-8c615a-openshift-default-5b86f7bbcf-kkrr8"
Jul 06 18:52:03.846881 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:03.846845    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-shared-a-8c615a-74cb758b8d-wx964"]
Jul 06 18:52:03.847442 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:03.847422    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ca751fc6-5463-4131-a1ae-d280da8530c7" containerName="istio-proxy"
Jul 06 18:52:03.847485 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:03.847447    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="ca751fc6-5463-4131-a1ae-d280da8530c7" containerName="istio-proxy"
Jul 06 18:52:03.851126 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:03.851103    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-8c615a-74cb758b8d-wx964"
Jul 06 18:52:03.853570 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:03.853548    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-shared-a-8c615a-serving-cert\""
Jul 06 18:52:03.861085 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:03.861065    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-8c615a-74cb758b8d-wx964"]
Jul 06 18:52:03.867665 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:03.867643    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/455f11a6-7dde-4dff-8aab-9faad1d6a318-maas-api-tls\") pod \"maas-api-e2e-shared-a-8c615a-74cb758b8d-wx964\" (UID: \"455f11a6-7dde-4dff-8aab-9faad1d6a318\") " pod="opendatahub/maas-api-e2e-shared-a-8c615a-74cb758b8d-wx964"
Jul 06 18:52:03.867851 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:03.867825    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pbn7v\" (UniqueName: \"kubernetes.io/projected/455f11a6-7dde-4dff-8aab-9faad1d6a318-kube-api-access-pbn7v\") pod \"maas-api-e2e-shared-a-8c615a-74cb758b8d-wx964\" (UID: \"455f11a6-7dde-4dff-8aab-9faad1d6a318\") " pod="opendatahub/maas-api-e2e-shared-a-8c615a-74cb758b8d-wx964"
Jul 06 18:52:03.867997 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:03.867893    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/455f11a6-7dde-4dff-8aab-9faad1d6a318-openshift-service-ca\") pod \"maas-api-e2e-shared-a-8c615a-74cb758b8d-wx964\" (UID: \"455f11a6-7dde-4dff-8aab-9faad1d6a318\") " pod="opendatahub/maas-api-e2e-shared-a-8c615a-74cb758b8d-wx964"
Jul 06 18:52:03.968818 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:03.968767    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/455f11a6-7dde-4dff-8aab-9faad1d6a318-maas-api-tls\") pod \"maas-api-e2e-shared-a-8c615a-74cb758b8d-wx964\" (UID: \"455f11a6-7dde-4dff-8aab-9faad1d6a318\") " pod="opendatahub/maas-api-e2e-shared-a-8c615a-74cb758b8d-wx964"
Jul 06 18:52:03.969007 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:03.968902    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pbn7v\" (UniqueName: \"kubernetes.io/projected/455f11a6-7dde-4dff-8aab-9faad1d6a318-kube-api-access-pbn7v\") pod \"maas-api-e2e-shared-a-8c615a-74cb758b8d-wx964\" (UID: \"455f11a6-7dde-4dff-8aab-9faad1d6a318\") " pod="opendatahub/maas-api-e2e-shared-a-8c615a-74cb758b8d-wx964"
Jul 06 18:52:03.969007 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:03.968969    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/455f11a6-7dde-4dff-8aab-9faad1d6a318-openshift-service-ca\") pod \"maas-api-e2e-shared-a-8c615a-74cb758b8d-wx964\" (UID: \"455f11a6-7dde-4dff-8aab-9faad1d6a318\") " pod="opendatahub/maas-api-e2e-shared-a-8c615a-74cb758b8d-wx964"
Jul 06 18:52:03.969251 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:52:03.969221    2580 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-shared-a-8c615a-serving-cert: secret "maas-api-e2e-shared-a-8c615a-serving-cert" not found
Jul 06 18:52:03.969327 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:52:03.969301    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/455f11a6-7dde-4dff-8aab-9faad1d6a318-maas-api-tls podName:455f11a6-7dde-4dff-8aab-9faad1d6a318 nodeName:}" failed. No retries permitted until 2026-07-06 18:52:04.46927925 +0000 UTC m=+2059.626677736 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/455f11a6-7dde-4dff-8aab-9faad1d6a318-maas-api-tls") pod "maas-api-e2e-shared-a-8c615a-74cb758b8d-wx964" (UID: "455f11a6-7dde-4dff-8aab-9faad1d6a318") : secret "maas-api-e2e-shared-a-8c615a-serving-cert" not found
Jul 06 18:52:03.969816 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:03.969790    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/455f11a6-7dde-4dff-8aab-9faad1d6a318-openshift-service-ca\") pod \"maas-api-e2e-shared-a-8c615a-74cb758b8d-wx964\" (UID: \"455f11a6-7dde-4dff-8aab-9faad1d6a318\") " pod="opendatahub/maas-api-e2e-shared-a-8c615a-74cb758b8d-wx964"
Jul 06 18:52:03.981820 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:03.981791    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pbn7v\" (UniqueName: \"kubernetes.io/projected/455f11a6-7dde-4dff-8aab-9faad1d6a318-kube-api-access-pbn7v\") pod \"maas-api-e2e-shared-a-8c615a-74cb758b8d-wx964\" (UID: \"455f11a6-7dde-4dff-8aab-9faad1d6a318\") " pod="opendatahub/maas-api-e2e-shared-a-8c615a-74cb758b8d-wx964"
Jul 06 18:52:04.474960 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:04.474924    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/455f11a6-7dde-4dff-8aab-9faad1d6a318-maas-api-tls\") pod \"maas-api-e2e-shared-a-8c615a-74cb758b8d-wx964\" (UID: \"455f11a6-7dde-4dff-8aab-9faad1d6a318\") " pod="opendatahub/maas-api-e2e-shared-a-8c615a-74cb758b8d-wx964"
Jul 06 18:52:04.477343 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:04.477322    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/455f11a6-7dde-4dff-8aab-9faad1d6a318-maas-api-tls\") pod \"maas-api-e2e-shared-a-8c615a-74cb758b8d-wx964\" (UID: \"455f11a6-7dde-4dff-8aab-9faad1d6a318\") " pod="opendatahub/maas-api-e2e-shared-a-8c615a-74cb758b8d-wx964"
Jul 06 18:52:04.764604 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:04.764517    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-8c615a-74cb758b8d-wx964"
Jul 06 18:52:04.908637 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:04.908612    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-8c615a-74cb758b8d-wx964"]
Jul 06 18:52:04.910765 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:52:04.910731    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod455f11a6_7dde_4dff_8aab_9faad1d6a318.slice/crio-0278a83f4a77f2b7ba493046be597e21953690a24a647d5b5566ff15e3c141b2 WatchSource:0}: Error finding container 0278a83f4a77f2b7ba493046be597e21953690a24a647d5b5566ff15e3c141b2: Status 404 returned error can't find the container with id 0278a83f4a77f2b7ba493046be597e21953690a24a647d5b5566ff15e3c141b2
Jul 06 18:52:05.448695 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:05.448626    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-13bc82-openshift-default-59754b7cbc-9wnkq"
Jul 06 18:52:05.486052 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:05.486008    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/600be5b1-255c-4e18-8ea4-9ab099e616aa-credential-socket\") pod \"600be5b1-255c-4e18-8ea4-9ab099e616aa\" (UID: \"600be5b1-255c-4e18-8ea4-9ab099e616aa\") "
Jul 06 18:52:05.486230 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:05.486065    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/600be5b1-255c-4e18-8ea4-9ab099e616aa-istio-envoy\") pod \"600be5b1-255c-4e18-8ea4-9ab099e616aa\" (UID: \"600be5b1-255c-4e18-8ea4-9ab099e616aa\") "
Jul 06 18:52:05.486230 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:05.486096    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/600be5b1-255c-4e18-8ea4-9ab099e616aa-istio-data\") pod \"600be5b1-255c-4e18-8ea4-9ab099e616aa\" (UID: \"600be5b1-255c-4e18-8ea4-9ab099e616aa\") "
Jul 06 18:52:05.486230 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:05.486158    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/600be5b1-255c-4e18-8ea4-9ab099e616aa-istiod-ca-cert\") pod \"600be5b1-255c-4e18-8ea4-9ab099e616aa\" (UID: \"600be5b1-255c-4e18-8ea4-9ab099e616aa\") "
Jul 06 18:52:05.486230 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:05.486190    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/600be5b1-255c-4e18-8ea4-9ab099e616aa-istio-podinfo\") pod \"600be5b1-255c-4e18-8ea4-9ab099e616aa\" (UID: \"600be5b1-255c-4e18-8ea4-9ab099e616aa\") "
Jul 06 18:52:05.486448 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:05.486277    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/600be5b1-255c-4e18-8ea4-9ab099e616aa-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "600be5b1-255c-4e18-8ea4-9ab099e616aa" (UID: "600be5b1-255c-4e18-8ea4-9ab099e616aa"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 18:52:05.486555 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:05.486245    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/600be5b1-255c-4e18-8ea4-9ab099e616aa-istio-token\") pod \"600be5b1-255c-4e18-8ea4-9ab099e616aa\" (UID: \"600be5b1-255c-4e18-8ea4-9ab099e616aa\") "
Jul 06 18:52:05.486659 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:05.486642    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/600be5b1-255c-4e18-8ea4-9ab099e616aa-workload-certs\") pod \"600be5b1-255c-4e18-8ea4-9ab099e616aa\" (UID: \"600be5b1-255c-4e18-8ea4-9ab099e616aa\") "
Jul 06 18:52:05.486722 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:05.486678    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/600be5b1-255c-4e18-8ea4-9ab099e616aa-workload-socket\") pod \"600be5b1-255c-4e18-8ea4-9ab099e616aa\" (UID: \"600be5b1-255c-4e18-8ea4-9ab099e616aa\") "
Jul 06 18:52:05.486773 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:05.486743    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vcvz6\" (UniqueName: \"kubernetes.io/projected/600be5b1-255c-4e18-8ea4-9ab099e616aa-kube-api-access-vcvz6\") pod \"600be5b1-255c-4e18-8ea4-9ab099e616aa\" (UID: \"600be5b1-255c-4e18-8ea4-9ab099e616aa\") "
Jul 06 18:52:05.486849 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:05.486550    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/600be5b1-255c-4e18-8ea4-9ab099e616aa-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "600be5b1-255c-4e18-8ea4-9ab099e616aa" (UID: "600be5b1-255c-4e18-8ea4-9ab099e616aa"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 18:52:05.486926 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:05.486902    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/600be5b1-255c-4e18-8ea4-9ab099e616aa-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "600be5b1-255c-4e18-8ea4-9ab099e616aa" (UID: "600be5b1-255c-4e18-8ea4-9ab099e616aa"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 18:52:05.487037 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:05.486988    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/600be5b1-255c-4e18-8ea4-9ab099e616aa-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "600be5b1-255c-4e18-8ea4-9ab099e616aa" (UID: "600be5b1-255c-4e18-8ea4-9ab099e616aa"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 18:52:05.487037 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:05.487004    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/600be5b1-255c-4e18-8ea4-9ab099e616aa-istio-data" (OuterVolumeSpecName: "istio-data") pod "600be5b1-255c-4e18-8ea4-9ab099e616aa" (UID: "600be5b1-255c-4e18-8ea4-9ab099e616aa"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 18:52:05.487363 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:05.487343    2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/600be5b1-255c-4e18-8ea4-9ab099e616aa-istiod-ca-cert\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:52:05.487443 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:05.487434    2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/600be5b1-255c-4e18-8ea4-9ab099e616aa-workload-certs\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:52:05.487530 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:05.487516    2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/600be5b1-255c-4e18-8ea4-9ab099e616aa-workload-socket\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:52:05.487584 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:05.487536    2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/600be5b1-255c-4e18-8ea4-9ab099e616aa-credential-socket\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:52:05.487584 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:05.487551    2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/600be5b1-255c-4e18-8ea4-9ab099e616aa-istio-data\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:52:05.488628 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:05.488601    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/600be5b1-255c-4e18-8ea4-9ab099e616aa-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "600be5b1-255c-4e18-8ea4-9ab099e616aa" (UID: "600be5b1-255c-4e18-8ea4-9ab099e616aa"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 18:52:05.488872 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:05.488853    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/600be5b1-255c-4e18-8ea4-9ab099e616aa-istio-token" (OuterVolumeSpecName: "istio-token") pod "600be5b1-255c-4e18-8ea4-9ab099e616aa" (UID: "600be5b1-255c-4e18-8ea4-9ab099e616aa"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 18:52:05.488986 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:05.488966    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/600be5b1-255c-4e18-8ea4-9ab099e616aa-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "600be5b1-255c-4e18-8ea4-9ab099e616aa" (UID: "600be5b1-255c-4e18-8ea4-9ab099e616aa"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 06 18:52:05.489295 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:05.489276    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/600be5b1-255c-4e18-8ea4-9ab099e616aa-kube-api-access-vcvz6" (OuterVolumeSpecName: "kube-api-access-vcvz6") pod "600be5b1-255c-4e18-8ea4-9ab099e616aa" (UID: "600be5b1-255c-4e18-8ea4-9ab099e616aa"). InnerVolumeSpecName "kube-api-access-vcvz6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 18:52:05.588199 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:05.588161    2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/600be5b1-255c-4e18-8ea4-9ab099e616aa-istio-envoy\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:52:05.588199 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:05.588193    2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/600be5b1-255c-4e18-8ea4-9ab099e616aa-istio-podinfo\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:52:05.588199 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:05.588202    2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/600be5b1-255c-4e18-8ea4-9ab099e616aa-istio-token\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:52:05.588429 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:05.588211    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vcvz6\" (UniqueName: \"kubernetes.io/projected/600be5b1-255c-4e18-8ea4-9ab099e616aa-kube-api-access-vcvz6\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:52:05.835544 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:05.835508    2580 generic.go:356] "Generic (PLEG): container finished" podID="600be5b1-255c-4e18-8ea4-9ab099e616aa" containerID="27ffe1c71e4802e4bcce487a20062c4d605ad9afdbd0e05b039d0e2182e2ed19" exitCode=0
Jul 06 18:52:05.835745 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:05.835559    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-13bc82-openshift-default-59754b7cbc-9wnkq"
Jul 06 18:52:05.835745 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:05.835552    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-13bc82-openshift-default-59754b7cbc-9wnkq" event={"ID":"600be5b1-255c-4e18-8ea4-9ab099e616aa","Type":"ContainerDied","Data":"27ffe1c71e4802e4bcce487a20062c4d605ad9afdbd0e05b039d0e2182e2ed19"}
Jul 06 18:52:05.835745 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:05.835691    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-13bc82-openshift-default-59754b7cbc-9wnkq" event={"ID":"600be5b1-255c-4e18-8ea4-9ab099e616aa","Type":"ContainerDied","Data":"c1e4e3004ee2aa003a4caac0637c1c0dbecd07e69d159122e14f13cfae6dba84"}
Jul 06 18:52:05.835745 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:05.835724    2580 scope.go:117] "RemoveContainer" containerID="27ffe1c71e4802e4bcce487a20062c4d605ad9afdbd0e05b039d0e2182e2ed19"
Jul 06 18:52:05.837306 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:05.837281    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-8c615a-74cb758b8d-wx964" event={"ID":"455f11a6-7dde-4dff-8aab-9faad1d6a318","Type":"ContainerStarted","Data":"50d1823fbb8619d3cf651e6a85e681728f512db5e656363e486abe95e07df5e0"}
Jul 06 18:52:05.837414 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:05.837323    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-8c615a-74cb758b8d-wx964" event={"ID":"455f11a6-7dde-4dff-8aab-9faad1d6a318","Type":"ContainerStarted","Data":"0278a83f4a77f2b7ba493046be597e21953690a24a647d5b5566ff15e3c141b2"}
Jul 06 18:52:05.837479 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:05.837455    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-shared-a-8c615a-74cb758b8d-wx964"
Jul 06 18:52:05.844747 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:05.844729    2580 scope.go:117] "RemoveContainer" containerID="41368300b8350c6c6dc5c2e3b41b04e04e5df448d4970003182416974ad358f4"
Jul 06 18:52:05.852626 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:05.852611    2580 scope.go:117] "RemoveContainer" containerID="27ffe1c71e4802e4bcce487a20062c4d605ad9afdbd0e05b039d0e2182e2ed19"
Jul 06 18:52:05.852870 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:52:05.852852    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"27ffe1c71e4802e4bcce487a20062c4d605ad9afdbd0e05b039d0e2182e2ed19\": container with ID starting with 27ffe1c71e4802e4bcce487a20062c4d605ad9afdbd0e05b039d0e2182e2ed19 not found: ID does not exist" containerID="27ffe1c71e4802e4bcce487a20062c4d605ad9afdbd0e05b039d0e2182e2ed19"
Jul 06 18:52:05.852914 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:05.852877    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"27ffe1c71e4802e4bcce487a20062c4d605ad9afdbd0e05b039d0e2182e2ed19"} err="failed to get container status \"27ffe1c71e4802e4bcce487a20062c4d605ad9afdbd0e05b039d0e2182e2ed19\": rpc error: code = NotFound desc = could not find container \"27ffe1c71e4802e4bcce487a20062c4d605ad9afdbd0e05b039d0e2182e2ed19\": container with ID starting with 27ffe1c71e4802e4bcce487a20062c4d605ad9afdbd0e05b039d0e2182e2ed19 not found: ID does not exist"
Jul 06 18:52:05.852914 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:05.852901    2580 scope.go:117] "RemoveContainer" containerID="41368300b8350c6c6dc5c2e3b41b04e04e5df448d4970003182416974ad358f4"
Jul 06 18:52:05.853148 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:52:05.853130    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"41368300b8350c6c6dc5c2e3b41b04e04e5df448d4970003182416974ad358f4\": container with ID starting with 41368300b8350c6c6dc5c2e3b41b04e04e5df448d4970003182416974ad358f4 not found: ID does not exist" containerID="41368300b8350c6c6dc5c2e3b41b04e04e5df448d4970003182416974ad358f4"
Jul 06 18:52:05.853207 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:05.853152    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"41368300b8350c6c6dc5c2e3b41b04e04e5df448d4970003182416974ad358f4"} err="failed to get container status \"41368300b8350c6c6dc5c2e3b41b04e04e5df448d4970003182416974ad358f4\": rpc error: code = NotFound desc = could not find container \"41368300b8350c6c6dc5c2e3b41b04e04e5df448d4970003182416974ad358f4\": container with ID starting with 41368300b8350c6c6dc5c2e3b41b04e04e5df448d4970003182416974ad358f4 not found: ID does not exist"
Jul 06 18:52:05.858642 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:05.858600    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-shared-a-8c615a-74cb758b8d-wx964" podStartSLOduration=2.546841595 podStartE2EDuration="2.858587319s" totalImagesPullingTime="311.745724ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:52:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:52:05.857333733 +0000 UTC m=+2061.014732218" watchObservedRunningTime="2026-07-06 18:52:05.858587319 +0000 UTC m=+2061.015985805"
Jul 06 18:52:05.874234 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:05.874207    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-13bc82-openshift-default-59754b7cbc-9wnkq"]
Jul 06 18:52:05.878222 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:05.878198    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-13bc82-openshift-default-59754b7cbc-9wnkq"]
Jul 06 18:52:07.323850 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:07.323749    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="600be5b1-255c-4e18-8ea4-9ab099e616aa" path="/var/lib/kubelet/pods/600be5b1-255c-4e18-8ea4-9ab099e616aa/volumes"
Jul 06 18:52:08.965625 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:08.965584    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-b67177-openshift-default-78b7697575-tz649"]
Jul 06 18:52:08.966172 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:08.966153    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="600be5b1-255c-4e18-8ea4-9ab099e616aa" containerName="istio-proxy"
Jul 06 18:52:08.966255 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:08.966175    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="600be5b1-255c-4e18-8ea4-9ab099e616aa" containerName="istio-proxy"
Jul 06 18:52:08.966255 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:08.966212    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="600be5b1-255c-4e18-8ea4-9ab099e616aa" containerName="istio-proxy"
Jul 06 18:52:08.966255 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:08.966220    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="600be5b1-255c-4e18-8ea4-9ab099e616aa" containerName="istio-proxy"
Jul 06 18:52:08.966365 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:08.966330    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="600be5b1-255c-4e18-8ea4-9ab099e616aa" containerName="istio-proxy"
Jul 06 18:52:08.971198 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:08.971177    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-b67177-openshift-default-78b7697575-tz649"
Jul 06 18:52:08.973940 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:08.973919    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-b67177-openshift-default-dockercfg-hkmt4\""
Jul 06 18:52:08.983747 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:08.983717    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-b67177-openshift-default-78b7697575-tz649"]
Jul 06 18:52:09.024815 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:09.024783    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p75rm\" (UniqueName: \"kubernetes.io/projected/782ee0c9-e518-4f88-a42d-73f6fc752038-kube-api-access-p75rm\") pod \"e2e-shared-b-b67177-openshift-default-78b7697575-tz649\" (UID: \"782ee0c9-e518-4f88-a42d-73f6fc752038\") " pod="openshift-ingress/e2e-shared-b-b67177-openshift-default-78b7697575-tz649"
Jul 06 18:52:09.024815 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:09.024820    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/782ee0c9-e518-4f88-a42d-73f6fc752038-workload-certs\") pod \"e2e-shared-b-b67177-openshift-default-78b7697575-tz649\" (UID: \"782ee0c9-e518-4f88-a42d-73f6fc752038\") " pod="openshift-ingress/e2e-shared-b-b67177-openshift-default-78b7697575-tz649"
Jul 06 18:52:09.025034 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:09.024859    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/782ee0c9-e518-4f88-a42d-73f6fc752038-istio-data\") pod \"e2e-shared-b-b67177-openshift-default-78b7697575-tz649\" (UID: \"782ee0c9-e518-4f88-a42d-73f6fc752038\") " pod="openshift-ingress/e2e-shared-b-b67177-openshift-default-78b7697575-tz649"
Jul 06 18:52:09.025034 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:09.024913    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/782ee0c9-e518-4f88-a42d-73f6fc752038-workload-socket\") pod \"e2e-shared-b-b67177-openshift-default-78b7697575-tz649\" (UID: \"782ee0c9-e518-4f88-a42d-73f6fc752038\") " pod="openshift-ingress/e2e-shared-b-b67177-openshift-default-78b7697575-tz649"
Jul 06 18:52:09.025034 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:09.024951    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/782ee0c9-e518-4f88-a42d-73f6fc752038-istio-podinfo\") pod \"e2e-shared-b-b67177-openshift-default-78b7697575-tz649\" (UID: \"782ee0c9-e518-4f88-a42d-73f6fc752038\") " pod="openshift-ingress/e2e-shared-b-b67177-openshift-default-78b7697575-tz649"
Jul 06 18:52:09.025034 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:09.024995    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/782ee0c9-e518-4f88-a42d-73f6fc752038-credential-socket\") pod \"e2e-shared-b-b67177-openshift-default-78b7697575-tz649\" (UID: \"782ee0c9-e518-4f88-a42d-73f6fc752038\") " pod="openshift-ingress/e2e-shared-b-b67177-openshift-default-78b7697575-tz649"
Jul 06 18:52:09.025177 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:09.025037    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/782ee0c9-e518-4f88-a42d-73f6fc752038-istiod-ca-cert\") pod \"e2e-shared-b-b67177-openshift-default-78b7697575-tz649\" (UID: \"782ee0c9-e518-4f88-a42d-73f6fc752038\") " pod="openshift-ingress/e2e-shared-b-b67177-openshift-default-78b7697575-tz649"
Jul 06 18:52:09.025177 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:09.025062    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/782ee0c9-e518-4f88-a42d-73f6fc752038-istio-token\") pod \"e2e-shared-b-b67177-openshift-default-78b7697575-tz649\" (UID: \"782ee0c9-e518-4f88-a42d-73f6fc752038\") " pod="openshift-ingress/e2e-shared-b-b67177-openshift-default-78b7697575-tz649"
Jul 06 18:52:09.025177 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:09.025115    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/782ee0c9-e518-4f88-a42d-73f6fc752038-istio-envoy\") pod \"e2e-shared-b-b67177-openshift-default-78b7697575-tz649\" (UID: \"782ee0c9-e518-4f88-a42d-73f6fc752038\") " pod="openshift-ingress/e2e-shared-b-b67177-openshift-default-78b7697575-tz649"
Jul 06 18:52:09.126680 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:09.126632    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p75rm\" (UniqueName: \"kubernetes.io/projected/782ee0c9-e518-4f88-a42d-73f6fc752038-kube-api-access-p75rm\") pod \"e2e-shared-b-b67177-openshift-default-78b7697575-tz649\" (UID: \"782ee0c9-e518-4f88-a42d-73f6fc752038\") " pod="openshift-ingress/e2e-shared-b-b67177-openshift-default-78b7697575-tz649"
Jul 06 18:52:09.126875 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:09.126686    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/782ee0c9-e518-4f88-a42d-73f6fc752038-workload-certs\") pod \"e2e-shared-b-b67177-openshift-default-78b7697575-tz649\" (UID: \"782ee0c9-e518-4f88-a42d-73f6fc752038\") " pod="openshift-ingress/e2e-shared-b-b67177-openshift-default-78b7697575-tz649"
Jul 06 18:52:09.126875 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:09.126716    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/782ee0c9-e518-4f88-a42d-73f6fc752038-istio-data\") pod \"e2e-shared-b-b67177-openshift-default-78b7697575-tz649\" (UID: \"782ee0c9-e518-4f88-a42d-73f6fc752038\") " pod="openshift-ingress/e2e-shared-b-b67177-openshift-default-78b7697575-tz649"
Jul 06 18:52:09.126875 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:09.126745    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/782ee0c9-e518-4f88-a42d-73f6fc752038-workload-socket\") pod \"e2e-shared-b-b67177-openshift-default-78b7697575-tz649\" (UID: \"782ee0c9-e518-4f88-a42d-73f6fc752038\") " pod="openshift-ingress/e2e-shared-b-b67177-openshift-default-78b7697575-tz649"
Jul 06 18:52:09.126875 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:09.126769    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/782ee0c9-e518-4f88-a42d-73f6fc752038-istio-podinfo\") pod \"e2e-shared-b-b67177-openshift-default-78b7697575-tz649\" (UID: \"782ee0c9-e518-4f88-a42d-73f6fc752038\") " pod="openshift-ingress/e2e-shared-b-b67177-openshift-default-78b7697575-tz649"
Jul 06 18:52:09.126875 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:09.126811    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/782ee0c9-e518-4f88-a42d-73f6fc752038-credential-socket\") pod \"e2e-shared-b-b67177-openshift-default-78b7697575-tz649\" (UID: \"782ee0c9-e518-4f88-a42d-73f6fc752038\") " pod="openshift-ingress/e2e-shared-b-b67177-openshift-default-78b7697575-tz649"
Jul 06 18:52:09.126875 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:09.126833    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/782ee0c9-e518-4f88-a42d-73f6fc752038-istiod-ca-cert\") pod \"e2e-shared-b-b67177-openshift-default-78b7697575-tz649\" (UID: \"782ee0c9-e518-4f88-a42d-73f6fc752038\") " pod="openshift-ingress/e2e-shared-b-b67177-openshift-default-78b7697575-tz649"
Jul 06 18:52:09.126875 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:09.126868    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/782ee0c9-e518-4f88-a42d-73f6fc752038-istio-token\") pod \"e2e-shared-b-b67177-openshift-default-78b7697575-tz649\" (UID: \"782ee0c9-e518-4f88-a42d-73f6fc752038\") " pod="openshift-ingress/e2e-shared-b-b67177-openshift-default-78b7697575-tz649"
Jul 06 18:52:09.127324 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:09.126915    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/782ee0c9-e518-4f88-a42d-73f6fc752038-istio-envoy\") pod \"e2e-shared-b-b67177-openshift-default-78b7697575-tz649\" (UID: \"782ee0c9-e518-4f88-a42d-73f6fc752038\") " pod="openshift-ingress/e2e-shared-b-b67177-openshift-default-78b7697575-tz649"
Jul 06 18:52:09.127762 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:09.127728    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/782ee0c9-e518-4f88-a42d-73f6fc752038-istio-data\") pod \"e2e-shared-b-b67177-openshift-default-78b7697575-tz649\" (UID: \"782ee0c9-e518-4f88-a42d-73f6fc752038\") " pod="openshift-ingress/e2e-shared-b-b67177-openshift-default-78b7697575-tz649"
Jul 06 18:52:09.127762 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:09.127747    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/782ee0c9-e518-4f88-a42d-73f6fc752038-workload-socket\") pod \"e2e-shared-b-b67177-openshift-default-78b7697575-tz649\" (UID: \"782ee0c9-e518-4f88-a42d-73f6fc752038\") " pod="openshift-ingress/e2e-shared-b-b67177-openshift-default-78b7697575-tz649"
Jul 06 18:52:09.127951 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:09.127799    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/782ee0c9-e518-4f88-a42d-73f6fc752038-workload-certs\") pod \"e2e-shared-b-b67177-openshift-default-78b7697575-tz649\" (UID: \"782ee0c9-e518-4f88-a42d-73f6fc752038\") " pod="openshift-ingress/e2e-shared-b-b67177-openshift-default-78b7697575-tz649"
Jul 06 18:52:09.128041 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:09.127951    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/782ee0c9-e518-4f88-a42d-73f6fc752038-credential-socket\") pod \"e2e-shared-b-b67177-openshift-default-78b7697575-tz649\" (UID: \"782ee0c9-e518-4f88-a42d-73f6fc752038\") " pod="openshift-ingress/e2e-shared-b-b67177-openshift-default-78b7697575-tz649"
Jul 06 18:52:09.128728 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:09.128700    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/782ee0c9-e518-4f88-a42d-73f6fc752038-istiod-ca-cert\") pod \"e2e-shared-b-b67177-openshift-default-78b7697575-tz649\" (UID: \"782ee0c9-e518-4f88-a42d-73f6fc752038\") " pod="openshift-ingress/e2e-shared-b-b67177-openshift-default-78b7697575-tz649"
Jul 06 18:52:09.129871 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:09.129845    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/782ee0c9-e518-4f88-a42d-73f6fc752038-istio-envoy\") pod \"e2e-shared-b-b67177-openshift-default-78b7697575-tz649\" (UID: \"782ee0c9-e518-4f88-a42d-73f6fc752038\") " pod="openshift-ingress/e2e-shared-b-b67177-openshift-default-78b7697575-tz649"
Jul 06 18:52:09.130446 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:09.130423    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/782ee0c9-e518-4f88-a42d-73f6fc752038-istio-podinfo\") pod \"e2e-shared-b-b67177-openshift-default-78b7697575-tz649\" (UID: \"782ee0c9-e518-4f88-a42d-73f6fc752038\") " pod="openshift-ingress/e2e-shared-b-b67177-openshift-default-78b7697575-tz649"
Jul 06 18:52:09.134615 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:09.134586    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/782ee0c9-e518-4f88-a42d-73f6fc752038-istio-token\") pod \"e2e-shared-b-b67177-openshift-default-78b7697575-tz649\" (UID: \"782ee0c9-e518-4f88-a42d-73f6fc752038\") " pod="openshift-ingress/e2e-shared-b-b67177-openshift-default-78b7697575-tz649"
Jul 06 18:52:09.134804 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:09.134773    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p75rm\" (UniqueName: \"kubernetes.io/projected/782ee0c9-e518-4f88-a42d-73f6fc752038-kube-api-access-p75rm\") pod \"e2e-shared-b-b67177-openshift-default-78b7697575-tz649\" (UID: \"782ee0c9-e518-4f88-a42d-73f6fc752038\") " pod="openshift-ingress/e2e-shared-b-b67177-openshift-default-78b7697575-tz649"
Jul 06 18:52:09.282346 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:09.282260    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-b67177-openshift-default-78b7697575-tz649"
Jul 06 18:52:09.423581 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:09.423544    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-b67177-openshift-default-78b7697575-tz649"]
Jul 06 18:52:09.433991 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:09.432788    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 06 18:52:09.433991 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:09.432875    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 06 18:52:09.433991 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:09.432922    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 06 18:52:09.858712 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:09.858671    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-b67177-openshift-default-78b7697575-tz649" event={"ID":"782ee0c9-e518-4f88-a42d-73f6fc752038","Type":"ContainerStarted","Data":"01ad046110af22074d68cebe3656a28cb6077d6b8c1cde8003097c46ee60818a"}
Jul 06 18:52:09.858901 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:09.858737    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-b67177-openshift-default-78b7697575-tz649" event={"ID":"782ee0c9-e518-4f88-a42d-73f6fc752038","Type":"ContainerStarted","Data":"fbe9b49175eabf52460393fc14df2c584955c522bed84977a0b5bf0d02e2e6ae"}
Jul 06 18:52:09.882622 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:09.882566    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-b67177-openshift-default-78b7697575-tz649" podStartSLOduration=1.882550022 podStartE2EDuration="1.882550022s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:52:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:52:09.879848651 +0000 UTC m=+2065.037247137" watchObservedRunningTime="2026-07-06 18:52:09.882550022 +0000 UTC m=+2065.039948509"
Jul 06 18:52:10.283461 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:10.283368    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-b67177-openshift-default-78b7697575-tz649"
Jul 06 18:52:10.288424 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:10.288398    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-b67177-openshift-default-78b7697575-tz649"
Jul 06 18:52:10.863773 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:10.863738    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-b67177-openshift-default-78b7697575-tz649"
Jul 06 18:52:10.864662 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:10.864641    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-b67177-openshift-default-78b7697575-tz649"
Jul 06 18:52:11.860857 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:11.860822    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-shared-a-8c615a-74cb758b8d-wx964"
Jul 06 18:52:11.946039 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:11.943964    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-shared-b-b67177-6dff4d8d49-zd4qr"]
Jul 06 18:52:11.946039 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:11.944765    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="600be5b1-255c-4e18-8ea4-9ab099e616aa" containerName="istio-proxy"
Jul 06 18:52:11.980070 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:11.980008    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-b67177-6dff4d8d49-zd4qr"]
Jul 06 18:52:11.980885 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:11.980160    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-b67177-6dff4d8d49-zd4qr"
Jul 06 18:52:11.983669 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:11.983642    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-shared-b-b67177-serving-cert\""
Jul 06 18:52:12.074721 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:12.074680    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fd743e06-bc6b-4891-97c9-b5eed07b14a9-openshift-service-ca\") pod \"maas-api-e2e-shared-b-b67177-6dff4d8d49-zd4qr\" (UID: \"fd743e06-bc6b-4891-97c9-b5eed07b14a9\") " pod="opendatahub/maas-api-e2e-shared-b-b67177-6dff4d8d49-zd4qr"
Jul 06 18:52:12.074987 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:12.074967    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x7ghb\" (UniqueName: \"kubernetes.io/projected/fd743e06-bc6b-4891-97c9-b5eed07b14a9-kube-api-access-x7ghb\") pod \"maas-api-e2e-shared-b-b67177-6dff4d8d49-zd4qr\" (UID: \"fd743e06-bc6b-4891-97c9-b5eed07b14a9\") " pod="opendatahub/maas-api-e2e-shared-b-b67177-6dff4d8d49-zd4qr"
Jul 06 18:52:12.075151 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:12.075135    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fd743e06-bc6b-4891-97c9-b5eed07b14a9-maas-api-tls\") pod \"maas-api-e2e-shared-b-b67177-6dff4d8d49-zd4qr\" (UID: \"fd743e06-bc6b-4891-97c9-b5eed07b14a9\") " pod="opendatahub/maas-api-e2e-shared-b-b67177-6dff4d8d49-zd4qr"
Jul 06 18:52:12.176572 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:12.176482    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fd743e06-bc6b-4891-97c9-b5eed07b14a9-openshift-service-ca\") pod \"maas-api-e2e-shared-b-b67177-6dff4d8d49-zd4qr\" (UID: \"fd743e06-bc6b-4891-97c9-b5eed07b14a9\") " pod="opendatahub/maas-api-e2e-shared-b-b67177-6dff4d8d49-zd4qr"
Jul 06 18:52:12.176572 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:12.176519    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x7ghb\" (UniqueName: \"kubernetes.io/projected/fd743e06-bc6b-4891-97c9-b5eed07b14a9-kube-api-access-x7ghb\") pod \"maas-api-e2e-shared-b-b67177-6dff4d8d49-zd4qr\" (UID: \"fd743e06-bc6b-4891-97c9-b5eed07b14a9\") " pod="opendatahub/maas-api-e2e-shared-b-b67177-6dff4d8d49-zd4qr"
Jul 06 18:52:12.176572 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:12.176544    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fd743e06-bc6b-4891-97c9-b5eed07b14a9-maas-api-tls\") pod \"maas-api-e2e-shared-b-b67177-6dff4d8d49-zd4qr\" (UID: \"fd743e06-bc6b-4891-97c9-b5eed07b14a9\") " pod="opendatahub/maas-api-e2e-shared-b-b67177-6dff4d8d49-zd4qr"
Jul 06 18:52:12.177150 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:12.177129    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fd743e06-bc6b-4891-97c9-b5eed07b14a9-openshift-service-ca\") pod \"maas-api-e2e-shared-b-b67177-6dff4d8d49-zd4qr\" (UID: \"fd743e06-bc6b-4891-97c9-b5eed07b14a9\") " pod="opendatahub/maas-api-e2e-shared-b-b67177-6dff4d8d49-zd4qr"
Jul 06 18:52:12.178978 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:12.178960    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fd743e06-bc6b-4891-97c9-b5eed07b14a9-maas-api-tls\") pod \"maas-api-e2e-shared-b-b67177-6dff4d8d49-zd4qr\" (UID: \"fd743e06-bc6b-4891-97c9-b5eed07b14a9\") " pod="opendatahub/maas-api-e2e-shared-b-b67177-6dff4d8d49-zd4qr"
Jul 06 18:52:12.185860 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:12.185837    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x7ghb\" (UniqueName: \"kubernetes.io/projected/fd743e06-bc6b-4891-97c9-b5eed07b14a9-kube-api-access-x7ghb\") pod \"maas-api-e2e-shared-b-b67177-6dff4d8d49-zd4qr\" (UID: \"fd743e06-bc6b-4891-97c9-b5eed07b14a9\") " pod="opendatahub/maas-api-e2e-shared-b-b67177-6dff4d8d49-zd4qr"
Jul 06 18:52:12.292137 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:12.292084    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-b67177-6dff4d8d49-zd4qr"
Jul 06 18:52:12.694814 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:12.694783    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-b67177-6dff4d8d49-zd4qr"]
Jul 06 18:52:12.696954 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:52:12.696928    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd743e06_bc6b_4891_97c9_b5eed07b14a9.slice/crio-8585b176b6e227eaf78073d48c89969e34d99765349ea75b60b4a936faea5074 WatchSource:0}: Error finding container 8585b176b6e227eaf78073d48c89969e34d99765349ea75b60b4a936faea5074: Status 404 returned error can't find the container with id 8585b176b6e227eaf78073d48c89969e34d99765349ea75b60b4a936faea5074
Jul 06 18:52:12.876421 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:12.876385    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-b67177-6dff4d8d49-zd4qr" event={"ID":"fd743e06-bc6b-4891-97c9-b5eed07b14a9","Type":"ContainerStarted","Data":"8585b176b6e227eaf78073d48c89969e34d99765349ea75b60b4a936faea5074"}
Jul 06 18:52:13.883825 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:13.883788    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-b67177-6dff4d8d49-zd4qr" event={"ID":"fd743e06-bc6b-4891-97c9-b5eed07b14a9","Type":"ContainerStarted","Data":"ddb047d13d864a0c84f21d8b9e05fe82454eb5b8f1a8e61b3852d2c92e53bcf5"}
Jul 06 18:52:13.884305 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:13.883887    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-shared-b-b67177-6dff4d8d49-zd4qr"
Jul 06 18:52:13.899673 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:13.899621    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-shared-b-b67177-6dff4d8d49-zd4qr" podStartSLOduration=2.387359325 podStartE2EDuration="2.899604852s" totalImagesPullingTime="512.245527ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:52:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:52:13.899554798 +0000 UTC m=+2069.056953285" watchObservedRunningTime="2026-07-06 18:52:13.899604852 +0000 UTC m=+2069.057003337"
Jul 06 18:52:16.902769 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:16.902734    2580 generic.go:356] "Generic (PLEG): container finished" podID="fef133e7-c5e3-4fa3-a6a5-f57239b3b56e" containerID="97616b491f19f13001a9a37cfb0cb797e1dd901959eeaea648c4cc903f160052" exitCode=0
Jul 06 18:52:16.903204 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:16.902807    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-8c615a-openshift-default-5b86f7bbcf-kkrr8" event={"ID":"fef133e7-c5e3-4fa3-a6a5-f57239b3b56e","Type":"ContainerDied","Data":"97616b491f19f13001a9a37cfb0cb797e1dd901959eeaea648c4cc903f160052"}
Jul 06 18:52:16.903254 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:16.903223    2580 scope.go:117] "RemoveContainer" containerID="97616b491f19f13001a9a37cfb0cb797e1dd901959eeaea648c4cc903f160052"
Jul 06 18:52:16.904287 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:16.904264    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 06 18:52:16.904357 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:16.904328    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 06 18:52:16.904397 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:16.904361    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 06 18:52:17.816773 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:17.816729    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-8c615a-openshift-default-5b86f7bbcf-kkrr8"
Jul 06 18:52:17.909796 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:17.909759    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-8c615a-openshift-default-5b86f7bbcf-kkrr8" event={"ID":"fef133e7-c5e3-4fa3-a6a5-f57239b3b56e","Type":"ContainerStarted","Data":"0bea27fa04868846d09f5609b681bd6c270d092c56f06e207c9984064a59a990"}
Jul 06 18:52:18.384442 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:18.384397    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-8c615a-openshift-default-5b86f7bbcf-kkrr8"
Jul 06 18:52:18.385662 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:18.385636    2580 patch_prober.go:28] interesting pod/e2e-shared-a-8c615a-openshift-default-5b86f7bbcf-kkrr8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body=
Jul 06 18:52:18.385725 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:18.385701    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-8c615a-openshift-default-5b86f7bbcf-kkrr8" podUID="fef133e7-c5e3-4fa3-a6a5-f57239b3b56e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused"
Jul 06 18:52:19.384088 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:19.384049    2580 patch_prober.go:28] interesting pod/e2e-shared-a-8c615a-openshift-default-5b86f7bbcf-kkrr8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body=
Jul 06 18:52:19.384483 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:19.384107    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-8c615a-openshift-default-5b86f7bbcf-kkrr8" podUID="fef133e7-c5e3-4fa3-a6a5-f57239b3b56e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused"
Jul 06 18:52:19.896959 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:19.896928    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-shared-b-b67177-6dff4d8d49-zd4qr"
Jul 06 18:52:20.384856 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:20.384796    2580 patch_prober.go:28] interesting pod/e2e-shared-a-8c615a-openshift-default-5b86f7bbcf-kkrr8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body=
Jul 06 18:52:20.385282 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:20.384876    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-8c615a-openshift-default-5b86f7bbcf-kkrr8" podUID="fef133e7-c5e3-4fa3-a6a5-f57239b3b56e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused"
Jul 06 18:52:21.384057 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:21.383989    2580 patch_prober.go:28] interesting pod/e2e-shared-a-8c615a-openshift-default-5b86f7bbcf-kkrr8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body=
Jul 06 18:52:21.384239 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:21.384079    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-8c615a-openshift-default-5b86f7bbcf-kkrr8" podUID="fef133e7-c5e3-4fa3-a6a5-f57239b3b56e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused"
Jul 06 18:52:22.388196 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:22.388166    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-8c615a-openshift-default-5b86f7bbcf-kkrr8"
Jul 06 18:52:22.388651 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:22.388574    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-8c615a-openshift-default-5b86f7bbcf-kkrr8"
Jul 06 18:52:22.389298 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:22.389279    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-8c615a-openshift-default-5b86f7bbcf-kkrr8"
Jul 06 18:52:26.606960 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:26.606920    2580 patch_prober.go:28] interesting pod/e2e-shared-b-b67177-openshift-default-78b7697575-tz649 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": read tcp 10.133.0.2:44102->10.133.0.69:15021: read: connection reset by peer" start-of-body=
Jul 06 18:52:26.607411 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:26.607001    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-b67177-openshift-default-78b7697575-tz649" podUID="782ee0c9-e518-4f88-a42d-73f6fc752038" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": read tcp 10.133.0.2:44102->10.133.0.69:15021: read: connection reset by peer"
Jul 06 18:52:26.961621 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:26.961529    2580 generic.go:356] "Generic (PLEG): container finished" podID="782ee0c9-e518-4f88-a42d-73f6fc752038" containerID="01ad046110af22074d68cebe3656a28cb6077d6b8c1cde8003097c46ee60818a" exitCode=0
Jul 06 18:52:26.961621 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:26.961598    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-b67177-openshift-default-78b7697575-tz649" event={"ID":"782ee0c9-e518-4f88-a42d-73f6fc752038","Type":"ContainerDied","Data":"01ad046110af22074d68cebe3656a28cb6077d6b8c1cde8003097c46ee60818a"}
Jul 06 18:52:26.962040 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:26.961998    2580 scope.go:117] "RemoveContainer" containerID="01ad046110af22074d68cebe3656a28cb6077d6b8c1cde8003097c46ee60818a"
Jul 06 18:52:26.963105 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:26.963077    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 06 18:52:26.963174 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:26.963145    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 06 18:52:26.963215 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:26.963175    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 06 18:52:27.969073 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:27.969035    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-b67177-openshift-default-78b7697575-tz649" event={"ID":"782ee0c9-e518-4f88-a42d-73f6fc752038","Type":"ContainerStarted","Data":"b1dfd3541e60c277d482c9e051f20430d3696819213ca9e1c4f116617a916475"}
Jul 06 18:52:28.282660 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:28.282567    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-b67177-openshift-default-78b7697575-tz649"
Jul 06 18:52:28.282891 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:28.282874    2580 patch_prober.go:28] interesting pod/e2e-shared-b-b67177-openshift-default-78b7697575-tz649 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body=
Jul 06 18:52:28.282954 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:28.282914    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-b67177-openshift-default-78b7697575-tz649" podUID="782ee0c9-e518-4f88-a42d-73f6fc752038" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused"
Jul 06 18:52:29.283611 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:29.283567    2580 patch_prober.go:28] interesting pod/e2e-shared-b-b67177-openshift-default-78b7697575-tz649 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body=
Jul 06 18:52:29.284049 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:29.283629    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-b67177-openshift-default-78b7697575-tz649" podUID="782ee0c9-e518-4f88-a42d-73f6fc752038" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused"
Jul 06 18:52:30.282780 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:30.282734    2580 patch_prober.go:28] interesting pod/e2e-shared-b-b67177-openshift-default-78b7697575-tz649 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body=
Jul 06 18:52:30.282967 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:30.282801    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-b67177-openshift-default-78b7697575-tz649" podUID="782ee0c9-e518-4f88-a42d-73f6fc752038" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused"
Jul 06 18:52:31.283184 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:31.283141    2580 patch_prober.go:28] interesting pod/e2e-shared-b-b67177-openshift-default-78b7697575-tz649 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body=
Jul 06 18:52:31.283672 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:31.283215    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-b67177-openshift-default-78b7697575-tz649" podUID="782ee0c9-e518-4f88-a42d-73f6fc752038" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused"
Jul 06 18:52:32.286838 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:32.286806    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-b67177-openshift-default-78b7697575-tz649"
Jul 06 18:52:32.287274 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:32.287078    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-b67177-openshift-default-78b7697575-tz649"
Jul 06 18:52:32.287808 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:52:32.287789    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-b67177-openshift-default-78b7697575-tz649"
Jul 06 18:54:53.994807 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:54:53.994770    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-8c615a/sub-test-model-8c615a-kserve-b56f5bf55-j565c"]
Jul 06 18:54:53.998258 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:54:53.998240    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-8c615a/sub-test-model-8c615a-kserve-b56f5bf55-j565c"
Jul 06 18:54:54.001174 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:54:54.001152    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-8c615a\"/\"kube-root-ca.crt\""
Jul 06 18:54:54.001312 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:54:54.001275    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-8c615a\"/\"openshift-service-ca.crt\""
Jul 06 18:54:54.002680 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:54:54.002663    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-8c615a\"/\"sub-test-model-8c615a-kserve-self-signed-certs\""
Jul 06 18:54:54.002848 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:54:54.002831    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-8c615a\"/\"default-dockercfg-s7fj4\""
Jul 06 18:54:54.008528 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:54:54.008505    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-8c615a/sub-test-model-8c615a-kserve-b56f5bf55-j565c"]
Jul 06 18:54:54.059027 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:54:54.058983    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z8vmw\" (UniqueName: \"kubernetes.io/projected/66a16c7d-03a3-4ab6-a5a4-4c5a17edd146-kube-api-access-z8vmw\") pod \"sub-test-model-8c615a-kserve-b56f5bf55-j565c\" (UID: \"66a16c7d-03a3-4ab6-a5a4-4c5a17edd146\") " pod="ai-tenant-e2e-shared-a-8c615a/sub-test-model-8c615a-kserve-b56f5bf55-j565c"
Jul 06 18:54:54.059224 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:54:54.059049    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/66a16c7d-03a3-4ab6-a5a4-4c5a17edd146-dshm\") pod \"sub-test-model-8c615a-kserve-b56f5bf55-j565c\" (UID: \"66a16c7d-03a3-4ab6-a5a4-4c5a17edd146\") " pod="ai-tenant-e2e-shared-a-8c615a/sub-test-model-8c615a-kserve-b56f5bf55-j565c"
Jul 06 18:54:54.059224 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:54:54.059069    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/66a16c7d-03a3-4ab6-a5a4-4c5a17edd146-tmp-dir\") pod \"sub-test-model-8c615a-kserve-b56f5bf55-j565c\" (UID: \"66a16c7d-03a3-4ab6-a5a4-4c5a17edd146\") " pod="ai-tenant-e2e-shared-a-8c615a/sub-test-model-8c615a-kserve-b56f5bf55-j565c"
Jul 06 18:54:54.059224 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:54:54.059131    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/66a16c7d-03a3-4ab6-a5a4-4c5a17edd146-model-cache\") pod \"sub-test-model-8c615a-kserve-b56f5bf55-j565c\" (UID: \"66a16c7d-03a3-4ab6-a5a4-4c5a17edd146\") " pod="ai-tenant-e2e-shared-a-8c615a/sub-test-model-8c615a-kserve-b56f5bf55-j565c"
Jul 06 18:54:54.059224 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:54:54.059170    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/66a16c7d-03a3-4ab6-a5a4-4c5a17edd146-tls-certs\") pod \"sub-test-model-8c615a-kserve-b56f5bf55-j565c\" (UID: \"66a16c7d-03a3-4ab6-a5a4-4c5a17edd146\") " pod="ai-tenant-e2e-shared-a-8c615a/sub-test-model-8c615a-kserve-b56f5bf55-j565c"
Jul 06 18:54:54.059415 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:54:54.059277    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/66a16c7d-03a3-4ab6-a5a4-4c5a17edd146-home\") pod \"sub-test-model-8c615a-kserve-b56f5bf55-j565c\" (UID: \"66a16c7d-03a3-4ab6-a5a4-4c5a17edd146\") " pod="ai-tenant-e2e-shared-a-8c615a/sub-test-model-8c615a-kserve-b56f5bf55-j565c"
Jul 06 18:54:54.159998 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:54:54.159958    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/66a16c7d-03a3-4ab6-a5a4-4c5a17edd146-dshm\") pod \"sub-test-model-8c615a-kserve-b56f5bf55-j565c\" (UID: \"66a16c7d-03a3-4ab6-a5a4-4c5a17edd146\") " pod="ai-tenant-e2e-shared-a-8c615a/sub-test-model-8c615a-kserve-b56f5bf55-j565c"
Jul 06 18:54:54.160205 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:54:54.160073    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/66a16c7d-03a3-4ab6-a5a4-4c5a17edd146-tmp-dir\") pod \"sub-test-model-8c615a-kserve-b56f5bf55-j565c\" (UID: \"66a16c7d-03a3-4ab6-a5a4-4c5a17edd146\") " pod="ai-tenant-e2e-shared-a-8c615a/sub-test-model-8c615a-kserve-b56f5bf55-j565c"
Jul 06 18:54:54.160205 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:54:54.160126    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/66a16c7d-03a3-4ab6-a5a4-4c5a17edd146-model-cache\") pod \"sub-test-model-8c615a-kserve-b56f5bf55-j565c\" (UID: \"66a16c7d-03a3-4ab6-a5a4-4c5a17edd146\") " pod="ai-tenant-e2e-shared-a-8c615a/sub-test-model-8c615a-kserve-b56f5bf55-j565c"
Jul 06 18:54:54.160205 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:54:54.160163    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/66a16c7d-03a3-4ab6-a5a4-4c5a17edd146-tls-certs\") pod \"sub-test-model-8c615a-kserve-b56f5bf55-j565c\" (UID: \"66a16c7d-03a3-4ab6-a5a4-4c5a17edd146\") " pod="ai-tenant-e2e-shared-a-8c615a/sub-test-model-8c615a-kserve-b56f5bf55-j565c"
Jul 06 18:54:54.160315 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:54:54.160260    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/66a16c7d-03a3-4ab6-a5a4-4c5a17edd146-home\") pod \"sub-test-model-8c615a-kserve-b56f5bf55-j565c\" (UID: \"66a16c7d-03a3-4ab6-a5a4-4c5a17edd146\") " pod="ai-tenant-e2e-shared-a-8c615a/sub-test-model-8c615a-kserve-b56f5bf55-j565c"
Jul 06 18:54:54.160374 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:54:54.160356    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z8vmw\" (UniqueName: \"kubernetes.io/projected/66a16c7d-03a3-4ab6-a5a4-4c5a17edd146-kube-api-access-z8vmw\") pod \"sub-test-model-8c615a-kserve-b56f5bf55-j565c\" (UID: \"66a16c7d-03a3-4ab6-a5a4-4c5a17edd146\") " pod="ai-tenant-e2e-shared-a-8c615a/sub-test-model-8c615a-kserve-b56f5bf55-j565c"
Jul 06 18:54:54.160492 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:54:54.160471    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/66a16c7d-03a3-4ab6-a5a4-4c5a17edd146-tmp-dir\") pod \"sub-test-model-8c615a-kserve-b56f5bf55-j565c\" (UID: \"66a16c7d-03a3-4ab6-a5a4-4c5a17edd146\") " pod="ai-tenant-e2e-shared-a-8c615a/sub-test-model-8c615a-kserve-b56f5bf55-j565c"
Jul 06 18:54:54.160550 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:54:54.160498    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/66a16c7d-03a3-4ab6-a5a4-4c5a17edd146-model-cache\") pod \"sub-test-model-8c615a-kserve-b56f5bf55-j565c\" (UID: \"66a16c7d-03a3-4ab6-a5a4-4c5a17edd146\") " pod="ai-tenant-e2e-shared-a-8c615a/sub-test-model-8c615a-kserve-b56f5bf55-j565c"
Jul 06 18:54:54.160634 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:54:54.160606    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/66a16c7d-03a3-4ab6-a5a4-4c5a17edd146-home\") pod \"sub-test-model-8c615a-kserve-b56f5bf55-j565c\" (UID: \"66a16c7d-03a3-4ab6-a5a4-4c5a17edd146\") " pod="ai-tenant-e2e-shared-a-8c615a/sub-test-model-8c615a-kserve-b56f5bf55-j565c"
Jul 06 18:54:54.162338 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:54:54.162317    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/66a16c7d-03a3-4ab6-a5a4-4c5a17edd146-dshm\") pod \"sub-test-model-8c615a-kserve-b56f5bf55-j565c\" (UID: \"66a16c7d-03a3-4ab6-a5a4-4c5a17edd146\") " pod="ai-tenant-e2e-shared-a-8c615a/sub-test-model-8c615a-kserve-b56f5bf55-j565c"
Jul 06 18:54:54.162617 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:54:54.162596    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/66a16c7d-03a3-4ab6-a5a4-4c5a17edd146-tls-certs\") pod \"sub-test-model-8c615a-kserve-b56f5bf55-j565c\" (UID: \"66a16c7d-03a3-4ab6-a5a4-4c5a17edd146\") " pod="ai-tenant-e2e-shared-a-8c615a/sub-test-model-8c615a-kserve-b56f5bf55-j565c"
Jul 06 18:54:54.169397 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:54:54.169373    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z8vmw\" (UniqueName: \"kubernetes.io/projected/66a16c7d-03a3-4ab6-a5a4-4c5a17edd146-kube-api-access-z8vmw\") pod \"sub-test-model-8c615a-kserve-b56f5bf55-j565c\" (UID: \"66a16c7d-03a3-4ab6-a5a4-4c5a17edd146\") " pod="ai-tenant-e2e-shared-a-8c615a/sub-test-model-8c615a-kserve-b56f5bf55-j565c"
Jul 06 18:54:54.308177 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:54:54.308073    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-8c615a/sub-test-model-8c615a-kserve-b56f5bf55-j565c"
Jul 06 18:54:54.448240 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:54:54.448214    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-8c615a/sub-test-model-8c615a-kserve-b56f5bf55-j565c"]
Jul 06 18:54:54.450261 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:54:54.450224    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod66a16c7d_03a3_4ab6_a5a4_4c5a17edd146.slice/crio-398126aff64431e5a16b764d5dcb99899178f5d8fd0f4dbdf30aa2285406ef75 WatchSource:0}: Error finding container 398126aff64431e5a16b764d5dcb99899178f5d8fd0f4dbdf30aa2285406ef75: Status 404 returned error can't find the container with id 398126aff64431e5a16b764d5dcb99899178f5d8fd0f4dbdf30aa2285406ef75
Jul 06 18:54:54.693222 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:54:54.693178    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-8c615a/sub-test-model-8c615a-kserve-b56f5bf55-j565c" event={"ID":"66a16c7d-03a3-4ab6-a5a4-4c5a17edd146","Type":"ContainerStarted","Data":"6cc001ac9393725f9b3679074dba568d56a84fcec48e7100057556b213a79c30"}
Jul 06 18:54:54.693406 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:54:54.693234    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-8c615a/sub-test-model-8c615a-kserve-b56f5bf55-j565c" event={"ID":"66a16c7d-03a3-4ab6-a5a4-4c5a17edd146","Type":"ContainerStarted","Data":"398126aff64431e5a16b764d5dcb99899178f5d8fd0f4dbdf30aa2285406ef75"}
Jul 06 18:54:54.712372 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:54:54.712311    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-8c615a/sub-test-model-8c615a-kserve-b56f5bf55-j565c" podStartSLOduration=1.7122971040000001 podStartE2EDuration="1.712297104s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:54:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:54:54.710415348 +0000 UTC m=+2229.867813826" watchObservedRunningTime="2026-07-06 18:54:54.712297104 +0000 UTC m=+2229.869695589"
Jul 06 18:55:04.309061 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:55:04.309001    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-8c615a/sub-test-model-8c615a-kserve-b56f5bf55-j565c"
Jul 06 18:55:04.309576 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:55:04.309093    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-8c615a/sub-test-model-8c615a-kserve-b56f5bf55-j565c"
Jul 06 18:55:04.321317 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:55:04.321289    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-8c615a/sub-test-model-8c615a-kserve-b56f5bf55-j565c"
Jul 06 18:55:04.756695 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:55:04.756660    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-8c615a/sub-test-model-8c615a-kserve-b56f5bf55-j565c"
Jul 06 18:55:51.631076 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:55:51.631036    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-8c615a/sub-test-model-8c615a-kserve-b56f5bf55-j565c"]
Jul 06 18:56:06.651099 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:56:06.651036    2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-8c615a/sub-test-model-8c615a-kserve-b56f5bf55-j565c" podUID="66a16c7d-03a3-4ab6-a5a4-4c5a17edd146" containerName="main" containerID="cri-o://6cc001ac9393725f9b3679074dba568d56a84fcec48e7100057556b213a79c30" gracePeriod=45
Jul 06 18:56:06.909244 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:56:06.909176    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-8c615a/sub-test-model-8c615a-kserve-b56f5bf55-j565c"
Jul 06 18:56:07.037196 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:56:07.037137    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/66a16c7d-03a3-4ab6-a5a4-4c5a17edd146-tls-certs\") pod \"66a16c7d-03a3-4ab6-a5a4-4c5a17edd146\" (UID: \"66a16c7d-03a3-4ab6-a5a4-4c5a17edd146\") "
Jul 06 18:56:07.037390 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:56:07.037243    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z8vmw\" (UniqueName: \"kubernetes.io/projected/66a16c7d-03a3-4ab6-a5a4-4c5a17edd146-kube-api-access-z8vmw\") pod \"66a16c7d-03a3-4ab6-a5a4-4c5a17edd146\" (UID: \"66a16c7d-03a3-4ab6-a5a4-4c5a17edd146\") "
Jul 06 18:56:07.037390 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:56:07.037277    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/66a16c7d-03a3-4ab6-a5a4-4c5a17edd146-tmp-dir\") pod \"66a16c7d-03a3-4ab6-a5a4-4c5a17edd146\" (UID: \"66a16c7d-03a3-4ab6-a5a4-4c5a17edd146\") "
Jul 06 18:56:07.037390 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:56:07.037308    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/66a16c7d-03a3-4ab6-a5a4-4c5a17edd146-model-cache\") pod \"66a16c7d-03a3-4ab6-a5a4-4c5a17edd146\" (UID: \"66a16c7d-03a3-4ab6-a5a4-4c5a17edd146\") "
Jul 06 18:56:07.037390 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:56:07.037337    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/66a16c7d-03a3-4ab6-a5a4-4c5a17edd146-dshm\") pod \"66a16c7d-03a3-4ab6-a5a4-4c5a17edd146\" (UID: \"66a16c7d-03a3-4ab6-a5a4-4c5a17edd146\") "
Jul 06 18:56:07.037578 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:56:07.037404    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/66a16c7d-03a3-4ab6-a5a4-4c5a17edd146-home\") pod \"66a16c7d-03a3-4ab6-a5a4-4c5a17edd146\" (UID: \"66a16c7d-03a3-4ab6-a5a4-4c5a17edd146\") "
Jul 06 18:56:07.037645 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:56:07.037577    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/66a16c7d-03a3-4ab6-a5a4-4c5a17edd146-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "66a16c7d-03a3-4ab6-a5a4-4c5a17edd146" (UID: "66a16c7d-03a3-4ab6-a5a4-4c5a17edd146"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 18:56:07.037645 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:56:07.037618    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/66a16c7d-03a3-4ab6-a5a4-4c5a17edd146-model-cache" (OuterVolumeSpecName: "model-cache") pod "66a16c7d-03a3-4ab6-a5a4-4c5a17edd146" (UID: "66a16c7d-03a3-4ab6-a5a4-4c5a17edd146"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 18:56:07.037782 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:56:07.037748    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/66a16c7d-03a3-4ab6-a5a4-4c5a17edd146-home" (OuterVolumeSpecName: "home") pod "66a16c7d-03a3-4ab6-a5a4-4c5a17edd146" (UID: "66a16c7d-03a3-4ab6-a5a4-4c5a17edd146"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 18:56:07.037782 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:56:07.037764    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/66a16c7d-03a3-4ab6-a5a4-4c5a17edd146-tmp-dir\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:56:07.037891 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:56:07.037803    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/66a16c7d-03a3-4ab6-a5a4-4c5a17edd146-model-cache\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:56:07.039550 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:56:07.039513    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/66a16c7d-03a3-4ab6-a5a4-4c5a17edd146-dshm" (OuterVolumeSpecName: "dshm") pod "66a16c7d-03a3-4ab6-a5a4-4c5a17edd146" (UID: "66a16c7d-03a3-4ab6-a5a4-4c5a17edd146"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 18:56:07.039683 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:56:07.039565    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/66a16c7d-03a3-4ab6-a5a4-4c5a17edd146-kube-api-access-z8vmw" (OuterVolumeSpecName: "kube-api-access-z8vmw") pod "66a16c7d-03a3-4ab6-a5a4-4c5a17edd146" (UID: "66a16c7d-03a3-4ab6-a5a4-4c5a17edd146"). InnerVolumeSpecName "kube-api-access-z8vmw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 18:56:07.039683 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:56:07.039618    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/66a16c7d-03a3-4ab6-a5a4-4c5a17edd146-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "66a16c7d-03a3-4ab6-a5a4-4c5a17edd146" (UID: "66a16c7d-03a3-4ab6-a5a4-4c5a17edd146"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 18:56:07.053695 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:56:07.053666    2580 generic.go:356] "Generic (PLEG): container finished" podID="66a16c7d-03a3-4ab6-a5a4-4c5a17edd146" containerID="6cc001ac9393725f9b3679074dba568d56a84fcec48e7100057556b213a79c30" exitCode=0
Jul 06 18:56:07.053803 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:56:07.053709    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-8c615a/sub-test-model-8c615a-kserve-b56f5bf55-j565c"
Jul 06 18:56:07.053803 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:56:07.053705    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-8c615a/sub-test-model-8c615a-kserve-b56f5bf55-j565c" event={"ID":"66a16c7d-03a3-4ab6-a5a4-4c5a17edd146","Type":"ContainerDied","Data":"6cc001ac9393725f9b3679074dba568d56a84fcec48e7100057556b213a79c30"}
Jul 06 18:56:07.053889 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:56:07.053817    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-8c615a/sub-test-model-8c615a-kserve-b56f5bf55-j565c" event={"ID":"66a16c7d-03a3-4ab6-a5a4-4c5a17edd146","Type":"ContainerDied","Data":"398126aff64431e5a16b764d5dcb99899178f5d8fd0f4dbdf30aa2285406ef75"}
Jul 06 18:56:07.053889 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:56:07.053841    2580 scope.go:117] "RemoveContainer" containerID="6cc001ac9393725f9b3679074dba568d56a84fcec48e7100057556b213a79c30"
Jul 06 18:56:07.062684 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:56:07.062663    2580 scope.go:117] "RemoveContainer" containerID="6cc001ac9393725f9b3679074dba568d56a84fcec48e7100057556b213a79c30"
Jul 06 18:56:07.062934 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:56:07.062913    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6cc001ac9393725f9b3679074dba568d56a84fcec48e7100057556b213a79c30\": container with ID starting with 6cc001ac9393725f9b3679074dba568d56a84fcec48e7100057556b213a79c30 not found: ID does not exist" containerID="6cc001ac9393725f9b3679074dba568d56a84fcec48e7100057556b213a79c30"
Jul 06 18:56:07.063006 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:56:07.062941    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6cc001ac9393725f9b3679074dba568d56a84fcec48e7100057556b213a79c30"} err="failed to get container status \"6cc001ac9393725f9b3679074dba568d56a84fcec48e7100057556b213a79c30\": rpc error: code = NotFound desc = could not find container \"6cc001ac9393725f9b3679074dba568d56a84fcec48e7100057556b213a79c30\": container with ID starting with 6cc001ac9393725f9b3679074dba568d56a84fcec48e7100057556b213a79c30 not found: ID does not exist"
Jul 06 18:56:07.076343 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:56:07.076321    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-8c615a/sub-test-model-8c615a-kserve-b56f5bf55-j565c"]
Jul 06 18:56:07.081274 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:56:07.081252    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-8c615a/sub-test-model-8c615a-kserve-b56f5bf55-j565c"]
Jul 06 18:56:07.138668 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:56:07.138635    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/66a16c7d-03a3-4ab6-a5a4-4c5a17edd146-tls-certs\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:56:07.138668 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:56:07.138663    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z8vmw\" (UniqueName: \"kubernetes.io/projected/66a16c7d-03a3-4ab6-a5a4-4c5a17edd146-kube-api-access-z8vmw\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:56:07.138668 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:56:07.138674    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/66a16c7d-03a3-4ab6-a5a4-4c5a17edd146-dshm\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:56:07.138893 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:56:07.138685    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/66a16c7d-03a3-4ab6-a5a4-4c5a17edd146-home\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:56:07.323565 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:56:07.323536    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="66a16c7d-03a3-4ab6-a5a4-4c5a17edd146" path="/var/lib/kubelet/pods/66a16c7d-03a3-4ab6-a5a4-4c5a17edd146/volumes"
Jul 06 18:56:50.697205 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:56:50.697167    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-b67177/rate-test-model-b67177-kserve-97fcb6867-4j276"]
Jul 06 18:56:50.697699 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:56:50.697585    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="66a16c7d-03a3-4ab6-a5a4-4c5a17edd146" containerName="main"
Jul 06 18:56:50.697699 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:56:50.697599    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="66a16c7d-03a3-4ab6-a5a4-4c5a17edd146" containerName="main"
Jul 06 18:56:50.697843 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:56:50.697715    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="66a16c7d-03a3-4ab6-a5a4-4c5a17edd146" containerName="main"
Jul 06 18:56:50.701057 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:56:50.701038    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-b67177/rate-test-model-b67177-kserve-97fcb6867-4j276"
Jul 06 18:56:50.703890 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:56:50.703859    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-b67177\"/\"default-dockercfg-7gff7\""
Jul 06 18:56:50.705077 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:56:50.705054    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-b67177\"/\"openshift-service-ca.crt\""
Jul 06 18:56:50.705198 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:56:50.705094    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-b67177\"/\"rate-test-model-b67177-kserve-self-signed-certs\""
Jul 06 18:56:50.705198 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:56:50.705103    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-b67177\"/\"kube-root-ca.crt\""
Jul 06 18:56:50.711850 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:56:50.711827    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-b67177/rate-test-model-b67177-kserve-97fcb6867-4j276"]
Jul 06 18:56:50.735740 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:56:50.735710    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8dad6f0a-b625-4617-a3d4-b55615c16401-home\") pod \"rate-test-model-b67177-kserve-97fcb6867-4j276\" (UID: \"8dad6f0a-b625-4617-a3d4-b55615c16401\") " pod="ai-tenant-e2e-shared-b-b67177/rate-test-model-b67177-kserve-97fcb6867-4j276"
Jul 06 18:56:50.735865 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:56:50.735752    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8dad6f0a-b625-4617-a3d4-b55615c16401-model-cache\") pod \"rate-test-model-b67177-kserve-97fcb6867-4j276\" (UID: \"8dad6f0a-b625-4617-a3d4-b55615c16401\") " pod="ai-tenant-e2e-shared-b-b67177/rate-test-model-b67177-kserve-97fcb6867-4j276"
Jul 06 18:56:50.735865 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:56:50.735778    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8dad6f0a-b625-4617-a3d4-b55615c16401-tls-certs\") pod \"rate-test-model-b67177-kserve-97fcb6867-4j276\" (UID: \"8dad6f0a-b625-4617-a3d4-b55615c16401\") " pod="ai-tenant-e2e-shared-b-b67177/rate-test-model-b67177-kserve-97fcb6867-4j276"
Jul 06 18:56:50.735865 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:56:50.735799    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8dad6f0a-b625-4617-a3d4-b55615c16401-dshm\") pod \"rate-test-model-b67177-kserve-97fcb6867-4j276\" (UID: \"8dad6f0a-b625-4617-a3d4-b55615c16401\") " pod="ai-tenant-e2e-shared-b-b67177/rate-test-model-b67177-kserve-97fcb6867-4j276"
Jul 06 18:56:50.735865 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:56:50.735822    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8dad6f0a-b625-4617-a3d4-b55615c16401-tmp-dir\") pod \"rate-test-model-b67177-kserve-97fcb6867-4j276\" (UID: \"8dad6f0a-b625-4617-a3d4-b55615c16401\") " pod="ai-tenant-e2e-shared-b-b67177/rate-test-model-b67177-kserve-97fcb6867-4j276"
Jul 06 18:56:50.735865 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:56:50.735839    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c69jm\" (UniqueName: \"kubernetes.io/projected/8dad6f0a-b625-4617-a3d4-b55615c16401-kube-api-access-c69jm\") pod \"rate-test-model-b67177-kserve-97fcb6867-4j276\" (UID: \"8dad6f0a-b625-4617-a3d4-b55615c16401\") " pod="ai-tenant-e2e-shared-b-b67177/rate-test-model-b67177-kserve-97fcb6867-4j276"
Jul 06 18:56:50.836424 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:56:50.836387    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8dad6f0a-b625-4617-a3d4-b55615c16401-tmp-dir\") pod \"rate-test-model-b67177-kserve-97fcb6867-4j276\" (UID: \"8dad6f0a-b625-4617-a3d4-b55615c16401\") " pod="ai-tenant-e2e-shared-b-b67177/rate-test-model-b67177-kserve-97fcb6867-4j276"
Jul 06 18:56:50.836612 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:56:50.836435    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c69jm\" (UniqueName: \"kubernetes.io/projected/8dad6f0a-b625-4617-a3d4-b55615c16401-kube-api-access-c69jm\") pod \"rate-test-model-b67177-kserve-97fcb6867-4j276\" (UID: \"8dad6f0a-b625-4617-a3d4-b55615c16401\") " pod="ai-tenant-e2e-shared-b-b67177/rate-test-model-b67177-kserve-97fcb6867-4j276"
Jul 06 18:56:50.836612 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:56:50.836490    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8dad6f0a-b625-4617-a3d4-b55615c16401-home\") pod \"rate-test-model-b67177-kserve-97fcb6867-4j276\" (UID: \"8dad6f0a-b625-4617-a3d4-b55615c16401\") " pod="ai-tenant-e2e-shared-b-b67177/rate-test-model-b67177-kserve-97fcb6867-4j276"
Jul 06 18:56:50.836612 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:56:50.836522    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8dad6f0a-b625-4617-a3d4-b55615c16401-model-cache\") pod \"rate-test-model-b67177-kserve-97fcb6867-4j276\" (UID: \"8dad6f0a-b625-4617-a3d4-b55615c16401\") " pod="ai-tenant-e2e-shared-b-b67177/rate-test-model-b67177-kserve-97fcb6867-4j276"
Jul 06 18:56:50.836612 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:56:50.836548    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8dad6f0a-b625-4617-a3d4-b55615c16401-tls-certs\") pod \"rate-test-model-b67177-kserve-97fcb6867-4j276\" (UID: \"8dad6f0a-b625-4617-a3d4-b55615c16401\") " pod="ai-tenant-e2e-shared-b-b67177/rate-test-model-b67177-kserve-97fcb6867-4j276"
Jul 06 18:56:50.836851 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:56:50.836673    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8dad6f0a-b625-4617-a3d4-b55615c16401-dshm\") pod \"rate-test-model-b67177-kserve-97fcb6867-4j276\" (UID: \"8dad6f0a-b625-4617-a3d4-b55615c16401\") " pod="ai-tenant-e2e-shared-b-b67177/rate-test-model-b67177-kserve-97fcb6867-4j276"
Jul 06 18:56:50.836851 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:56:50.836804    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8dad6f0a-b625-4617-a3d4-b55615c16401-tmp-dir\") pod \"rate-test-model-b67177-kserve-97fcb6867-4j276\" (UID: \"8dad6f0a-b625-4617-a3d4-b55615c16401\") " pod="ai-tenant-e2e-shared-b-b67177/rate-test-model-b67177-kserve-97fcb6867-4j276"
Jul 06 18:56:50.836944 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:56:50.836893    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8dad6f0a-b625-4617-a3d4-b55615c16401-model-cache\") pod \"rate-test-model-b67177-kserve-97fcb6867-4j276\" (UID: \"8dad6f0a-b625-4617-a3d4-b55615c16401\") " pod="ai-tenant-e2e-shared-b-b67177/rate-test-model-b67177-kserve-97fcb6867-4j276"
Jul 06 18:56:50.837032 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:56:50.836992    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8dad6f0a-b625-4617-a3d4-b55615c16401-home\") pod \"rate-test-model-b67177-kserve-97fcb6867-4j276\" (UID: \"8dad6f0a-b625-4617-a3d4-b55615c16401\") " pod="ai-tenant-e2e-shared-b-b67177/rate-test-model-b67177-kserve-97fcb6867-4j276"
Jul 06 18:56:50.838967 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:56:50.838937    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8dad6f0a-b625-4617-a3d4-b55615c16401-dshm\") pod \"rate-test-model-b67177-kserve-97fcb6867-4j276\" (UID: \"8dad6f0a-b625-4617-a3d4-b55615c16401\") " pod="ai-tenant-e2e-shared-b-b67177/rate-test-model-b67177-kserve-97fcb6867-4j276"
Jul 06 18:56:50.839212 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:56:50.839194    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8dad6f0a-b625-4617-a3d4-b55615c16401-tls-certs\") pod \"rate-test-model-b67177-kserve-97fcb6867-4j276\" (UID: \"8dad6f0a-b625-4617-a3d4-b55615c16401\") " pod="ai-tenant-e2e-shared-b-b67177/rate-test-model-b67177-kserve-97fcb6867-4j276"
Jul 06 18:56:50.847494 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:56:50.847469    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c69jm\" (UniqueName: \"kubernetes.io/projected/8dad6f0a-b625-4617-a3d4-b55615c16401-kube-api-access-c69jm\") pod \"rate-test-model-b67177-kserve-97fcb6867-4j276\" (UID: \"8dad6f0a-b625-4617-a3d4-b55615c16401\") " pod="ai-tenant-e2e-shared-b-b67177/rate-test-model-b67177-kserve-97fcb6867-4j276"
Jul 06 18:56:51.011575 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:56:51.011473    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-b67177/rate-test-model-b67177-kserve-97fcb6867-4j276"
Jul 06 18:56:51.150592 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:56:51.150566    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-b67177/rate-test-model-b67177-kserve-97fcb6867-4j276"]
Jul 06 18:56:51.151428 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:56:51.151404    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8dad6f0a_b625_4617_a3d4_b55615c16401.slice/crio-2c7d9a0f77460bf0da50a394b120426f0ee832e97f41a106c9cc4425d34801f8 WatchSource:0}: Error finding container 2c7d9a0f77460bf0da50a394b120426f0ee832e97f41a106c9cc4425d34801f8: Status 404 returned error can't find the container with id 2c7d9a0f77460bf0da50a394b120426f0ee832e97f41a106c9cc4425d34801f8
Jul 06 18:56:51.279995 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:56:51.279913    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-b67177/rate-test-model-b67177-kserve-97fcb6867-4j276" event={"ID":"8dad6f0a-b625-4617-a3d4-b55615c16401","Type":"ContainerStarted","Data":"434a9fa3b494e0efbbdde49136fc0c878834d61729d7d5796d3a0875a1286ef4"}
Jul 06 18:56:51.279995 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:56:51.279950    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-b67177/rate-test-model-b67177-kserve-97fcb6867-4j276" event={"ID":"8dad6f0a-b625-4617-a3d4-b55615c16401","Type":"ContainerStarted","Data":"2c7d9a0f77460bf0da50a394b120426f0ee832e97f41a106c9cc4425d34801f8"}
Jul 06 18:56:51.303101 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:56:51.302992    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-b67177/rate-test-model-b67177-kserve-97fcb6867-4j276" podStartSLOduration=1.302973602 podStartE2EDuration="1.302973602s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:56:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:56:51.301763039 +0000 UTC m=+2346.459161528" watchObservedRunningTime="2026-07-06 18:56:51.302973602 +0000 UTC m=+2346.460372090"
Jul 06 18:57:01.012708 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:57:01.012668    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-b67177/rate-test-model-b67177-kserve-97fcb6867-4j276"
Jul 06 18:57:01.012708 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:57:01.012718    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-b67177/rate-test-model-b67177-kserve-97fcb6867-4j276"
Jul 06 18:57:01.024932 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:57:01.024907    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-b67177/rate-test-model-b67177-kserve-97fcb6867-4j276"
Jul 06 18:57:01.344967 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:57:01.344937    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-b67177/rate-test-model-b67177-kserve-97fcb6867-4j276"
Jul 06 18:57:37.395270 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:57:37.395235    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-b67177/rate-test-model-b67177-kserve-97fcb6867-4j276"]
Jul 06 18:57:39.228919 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:57:39.228873    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-8c615a-74cb758b8d-wx964"]
Jul 06 18:57:39.229421 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:57:39.229204    2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-shared-a-8c615a-74cb758b8d-wx964" podUID="455f11a6-7dde-4dff-8aab-9faad1d6a318" containerName="maas-api" containerID="cri-o://50d1823fbb8619d3cf651e6a85e681728f512db5e656363e486abe95e07df5e0" gracePeriod=30
Jul 06 18:57:39.666537 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:57:39.666513    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-8c615a-74cb758b8d-wx964"
Jul 06 18:57:39.792335 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:57:39.792235    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/455f11a6-7dde-4dff-8aab-9faad1d6a318-maas-api-tls\") pod \"455f11a6-7dde-4dff-8aab-9faad1d6a318\" (UID: \"455f11a6-7dde-4dff-8aab-9faad1d6a318\") "
Jul 06 18:57:39.792514 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:57:39.792362    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pbn7v\" (UniqueName: \"kubernetes.io/projected/455f11a6-7dde-4dff-8aab-9faad1d6a318-kube-api-access-pbn7v\") pod \"455f11a6-7dde-4dff-8aab-9faad1d6a318\" (UID: \"455f11a6-7dde-4dff-8aab-9faad1d6a318\") "
Jul 06 18:57:39.792514 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:57:39.792383    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/455f11a6-7dde-4dff-8aab-9faad1d6a318-openshift-service-ca\") pod \"455f11a6-7dde-4dff-8aab-9faad1d6a318\" (UID: \"455f11a6-7dde-4dff-8aab-9faad1d6a318\") "
Jul 06 18:57:39.792809 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:57:39.792782    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/455f11a6-7dde-4dff-8aab-9faad1d6a318-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "455f11a6-7dde-4dff-8aab-9faad1d6a318" (UID: "455f11a6-7dde-4dff-8aab-9faad1d6a318"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 18:57:39.794482 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:57:39.794450    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/455f11a6-7dde-4dff-8aab-9faad1d6a318-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "455f11a6-7dde-4dff-8aab-9faad1d6a318" (UID: "455f11a6-7dde-4dff-8aab-9faad1d6a318"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 18:57:39.794595 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:57:39.794510    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/455f11a6-7dde-4dff-8aab-9faad1d6a318-kube-api-access-pbn7v" (OuterVolumeSpecName: "kube-api-access-pbn7v") pod "455f11a6-7dde-4dff-8aab-9faad1d6a318" (UID: "455f11a6-7dde-4dff-8aab-9faad1d6a318"). InnerVolumeSpecName "kube-api-access-pbn7v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 18:57:39.893362 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:57:39.893323    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pbn7v\" (UniqueName: \"kubernetes.io/projected/455f11a6-7dde-4dff-8aab-9faad1d6a318-kube-api-access-pbn7v\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:57:39.893362 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:57:39.893355    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/455f11a6-7dde-4dff-8aab-9faad1d6a318-openshift-service-ca\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:57:39.893362 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:57:39.893368    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/455f11a6-7dde-4dff-8aab-9faad1d6a318-maas-api-tls\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:57:40.533362 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:57:40.533329    2580 generic.go:356] "Generic (PLEG): container finished" podID="455f11a6-7dde-4dff-8aab-9faad1d6a318" containerID="50d1823fbb8619d3cf651e6a85e681728f512db5e656363e486abe95e07df5e0" exitCode=0
Jul 06 18:57:40.533794 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:57:40.533378    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-8c615a-74cb758b8d-wx964"
Jul 06 18:57:40.533794 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:57:40.533413    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-8c615a-74cb758b8d-wx964" event={"ID":"455f11a6-7dde-4dff-8aab-9faad1d6a318","Type":"ContainerDied","Data":"50d1823fbb8619d3cf651e6a85e681728f512db5e656363e486abe95e07df5e0"}
Jul 06 18:57:40.533794 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:57:40.533474    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-8c615a-74cb758b8d-wx964" event={"ID":"455f11a6-7dde-4dff-8aab-9faad1d6a318","Type":"ContainerDied","Data":"0278a83f4a77f2b7ba493046be597e21953690a24a647d5b5566ff15e3c141b2"}
Jul 06 18:57:40.533794 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:57:40.533503    2580 scope.go:117] "RemoveContainer" containerID="50d1823fbb8619d3cf651e6a85e681728f512db5e656363e486abe95e07df5e0"
Jul 06 18:57:40.543208 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:57:40.543192    2580 scope.go:117] "RemoveContainer" containerID="50d1823fbb8619d3cf651e6a85e681728f512db5e656363e486abe95e07df5e0"
Jul 06 18:57:40.543459 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:57:40.543440    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"50d1823fbb8619d3cf651e6a85e681728f512db5e656363e486abe95e07df5e0\": container with ID starting with 50d1823fbb8619d3cf651e6a85e681728f512db5e656363e486abe95e07df5e0 not found: ID does not exist" containerID="50d1823fbb8619d3cf651e6a85e681728f512db5e656363e486abe95e07df5e0"
Jul 06 18:57:40.543512 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:57:40.543469    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"50d1823fbb8619d3cf651e6a85e681728f512db5e656363e486abe95e07df5e0"} err="failed to get container status \"50d1823fbb8619d3cf651e6a85e681728f512db5e656363e486abe95e07df5e0\": rpc error: code = NotFound desc = could not find container \"50d1823fbb8619d3cf651e6a85e681728f512db5e656363e486abe95e07df5e0\": container with ID starting with 50d1823fbb8619d3cf651e6a85e681728f512db5e656363e486abe95e07df5e0 not found: ID does not exist"
Jul 06 18:57:40.559199 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:57:40.559171    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-8c615a-74cb758b8d-wx964"]
Jul 06 18:57:40.563337 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:57:40.563310    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-8c615a-74cb758b8d-wx964"]
Jul 06 18:57:41.323683 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:57:41.323647    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="455f11a6-7dde-4dff-8aab-9faad1d6a318" path="/var/lib/kubelet/pods/455f11a6-7dde-4dff-8aab-9faad1d6a318/volumes"
Jul 06 18:57:52.415800 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:57:52.415753    2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-b67177/rate-test-model-b67177-kserve-97fcb6867-4j276" podUID="8dad6f0a-b625-4617-a3d4-b55615c16401" containerName="main" containerID="cri-o://434a9fa3b494e0efbbdde49136fc0c878834d61729d7d5796d3a0875a1286ef4" gracePeriod=45
Jul 06 18:57:52.598654 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:57:52.598622    2580 generic.go:356] "Generic (PLEG): container finished" podID="8dad6f0a-b625-4617-a3d4-b55615c16401" containerID="434a9fa3b494e0efbbdde49136fc0c878834d61729d7d5796d3a0875a1286ef4" exitCode=0
Jul 06 18:57:52.598794 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:57:52.598676    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-b67177/rate-test-model-b67177-kserve-97fcb6867-4j276" event={"ID":"8dad6f0a-b625-4617-a3d4-b55615c16401","Type":"ContainerDied","Data":"434a9fa3b494e0efbbdde49136fc0c878834d61729d7d5796d3a0875a1286ef4"}
Jul 06 18:57:52.680100 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:57:52.680027    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-b67177/rate-test-model-b67177-kserve-97fcb6867-4j276"
Jul 06 18:57:52.706238 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:57:52.706208    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8dad6f0a-b625-4617-a3d4-b55615c16401-tls-certs\") pod \"8dad6f0a-b625-4617-a3d4-b55615c16401\" (UID: \"8dad6f0a-b625-4617-a3d4-b55615c16401\") "
Jul 06 18:57:52.706238 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:57:52.706244    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8dad6f0a-b625-4617-a3d4-b55615c16401-home\") pod \"8dad6f0a-b625-4617-a3d4-b55615c16401\" (UID: \"8dad6f0a-b625-4617-a3d4-b55615c16401\") "
Jul 06 18:57:52.706474 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:57:52.706270    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8dad6f0a-b625-4617-a3d4-b55615c16401-model-cache\") pod \"8dad6f0a-b625-4617-a3d4-b55615c16401\" (UID: \"8dad6f0a-b625-4617-a3d4-b55615c16401\") "
Jul 06 18:57:52.706474 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:57:52.706290    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8dad6f0a-b625-4617-a3d4-b55615c16401-tmp-dir\") pod \"8dad6f0a-b625-4617-a3d4-b55615c16401\" (UID: \"8dad6f0a-b625-4617-a3d4-b55615c16401\") "
Jul 06 18:57:52.706474 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:57:52.706328    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c69jm\" (UniqueName: \"kubernetes.io/projected/8dad6f0a-b625-4617-a3d4-b55615c16401-kube-api-access-c69jm\") pod \"8dad6f0a-b625-4617-a3d4-b55615c16401\" (UID: \"8dad6f0a-b625-4617-a3d4-b55615c16401\") "
Jul 06 18:57:52.706474 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:57:52.706370    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8dad6f0a-b625-4617-a3d4-b55615c16401-dshm\") pod \"8dad6f0a-b625-4617-a3d4-b55615c16401\" (UID: \"8dad6f0a-b625-4617-a3d4-b55615c16401\") "
Jul 06 18:57:52.706680 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:57:52.706571    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8dad6f0a-b625-4617-a3d4-b55615c16401-home" (OuterVolumeSpecName: "home") pod "8dad6f0a-b625-4617-a3d4-b55615c16401" (UID: "8dad6f0a-b625-4617-a3d4-b55615c16401"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 18:57:52.706680 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:57:52.706635    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8dad6f0a-b625-4617-a3d4-b55615c16401-model-cache" (OuterVolumeSpecName: "model-cache") pod "8dad6f0a-b625-4617-a3d4-b55615c16401" (UID: "8dad6f0a-b625-4617-a3d4-b55615c16401"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 18:57:52.706795 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:57:52.706733    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8dad6f0a-b625-4617-a3d4-b55615c16401-home\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:57:52.706795 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:57:52.706752    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8dad6f0a-b625-4617-a3d4-b55615c16401-model-cache\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:57:52.706886 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:57:52.706845    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8dad6f0a-b625-4617-a3d4-b55615c16401-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "8dad6f0a-b625-4617-a3d4-b55615c16401" (UID: "8dad6f0a-b625-4617-a3d4-b55615c16401"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 18:57:52.708722 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:57:52.708686    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8dad6f0a-b625-4617-a3d4-b55615c16401-dshm" (OuterVolumeSpecName: "dshm") pod "8dad6f0a-b625-4617-a3d4-b55615c16401" (UID: "8dad6f0a-b625-4617-a3d4-b55615c16401"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 18:57:52.708840 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:57:52.708742    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8dad6f0a-b625-4617-a3d4-b55615c16401-kube-api-access-c69jm" (OuterVolumeSpecName: "kube-api-access-c69jm") pod "8dad6f0a-b625-4617-a3d4-b55615c16401" (UID: "8dad6f0a-b625-4617-a3d4-b55615c16401"). InnerVolumeSpecName "kube-api-access-c69jm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 18:57:52.708884 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:57:52.708851    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8dad6f0a-b625-4617-a3d4-b55615c16401-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "8dad6f0a-b625-4617-a3d4-b55615c16401" (UID: "8dad6f0a-b625-4617-a3d4-b55615c16401"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 18:57:52.807374 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:57:52.807327    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8dad6f0a-b625-4617-a3d4-b55615c16401-tls-certs\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:57:52.807374 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:57:52.807368    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8dad6f0a-b625-4617-a3d4-b55615c16401-tmp-dir\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:57:52.807374 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:57:52.807380    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c69jm\" (UniqueName: \"kubernetes.io/projected/8dad6f0a-b625-4617-a3d4-b55615c16401-kube-api-access-c69jm\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:57:52.807374 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:57:52.807389    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8dad6f0a-b625-4617-a3d4-b55615c16401-dshm\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:57:53.604952 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:57:53.604866    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-b67177/rate-test-model-b67177-kserve-97fcb6867-4j276"
Jul 06 18:57:53.604952 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:57:53.604885    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-b67177/rate-test-model-b67177-kserve-97fcb6867-4j276" event={"ID":"8dad6f0a-b625-4617-a3d4-b55615c16401","Type":"ContainerDied","Data":"2c7d9a0f77460bf0da50a394b120426f0ee832e97f41a106c9cc4425d34801f8"}
Jul 06 18:57:53.604952 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:57:53.604931    2580 scope.go:117] "RemoveContainer" containerID="434a9fa3b494e0efbbdde49136fc0c878834d61729d7d5796d3a0875a1286ef4"
Jul 06 18:57:53.625483 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:57:53.625455    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-b67177/rate-test-model-b67177-kserve-97fcb6867-4j276"]
Jul 06 18:57:53.628393 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:57:53.628359    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-b67177/rate-test-model-b67177-kserve-97fcb6867-4j276"]
Jul 06 18:57:55.323845 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:57:55.323802    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8dad6f0a-b625-4617-a3d4-b55615c16401" path="/var/lib/kubelet/pods/8dad6f0a-b625-4617-a3d4-b55615c16401/volumes"
Jul 06 18:58:21.379606 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:21.379519    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-8c615a-openshift-default-5b86f7bbcf-kkrr8"]
Jul 06 18:58:21.380126 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:21.379809    2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-8c615a-openshift-default-5b86f7bbcf-kkrr8" podUID="fef133e7-c5e3-4fa3-a6a5-f57239b3b56e" containerName="istio-proxy" containerID="cri-o://0bea27fa04868846d09f5609b681bd6c270d092c56f06e207c9984064a59a990" gracePeriod=30
Jul 06 18:58:22.391277 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:22.391236    2580 patch_prober.go:28] interesting pod/e2e-shared-a-8c615a-openshift-default-5b86f7bbcf-kkrr8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 06 18:58:22.391657 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:22.391306    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-8c615a-openshift-default-5b86f7bbcf-kkrr8" podUID="fef133e7-c5e3-4fa3-a6a5-f57239b3b56e" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 06 18:58:22.604386 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:22.604356    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-b67177-6dff4d8d49-zd4qr"]
Jul 06 18:58:22.604601 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:22.604560    2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-shared-b-b67177-6dff4d8d49-zd4qr" podUID="fd743e06-bc6b-4891-97c9-b5eed07b14a9" containerName="maas-api" containerID="cri-o://ddb047d13d864a0c84f21d8b9e05fe82454eb5b8f1a8e61b3852d2c92e53bcf5" gracePeriod=30
Jul 06 18:58:22.747862 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:22.747830    2580 generic.go:356] "Generic (PLEG): container finished" podID="fd743e06-bc6b-4891-97c9-b5eed07b14a9" containerID="ddb047d13d864a0c84f21d8b9e05fe82454eb5b8f1a8e61b3852d2c92e53bcf5" exitCode=0
Jul 06 18:58:22.748025 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:22.747902    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-b67177-6dff4d8d49-zd4qr" event={"ID":"fd743e06-bc6b-4891-97c9-b5eed07b14a9","Type":"ContainerDied","Data":"ddb047d13d864a0c84f21d8b9e05fe82454eb5b8f1a8e61b3852d2c92e53bcf5"}
Jul 06 18:58:22.870009 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:22.869980    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-b67177-6dff4d8d49-zd4qr"
Jul 06 18:58:22.988800 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:22.988714    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fd743e06-bc6b-4891-97c9-b5eed07b14a9-openshift-service-ca\") pod \"fd743e06-bc6b-4891-97c9-b5eed07b14a9\" (UID: \"fd743e06-bc6b-4891-97c9-b5eed07b14a9\") "
Jul 06 18:58:22.988800 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:22.988759    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fd743e06-bc6b-4891-97c9-b5eed07b14a9-maas-api-tls\") pod \"fd743e06-bc6b-4891-97c9-b5eed07b14a9\" (UID: \"fd743e06-bc6b-4891-97c9-b5eed07b14a9\") "
Jul 06 18:58:22.988800 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:22.988800    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x7ghb\" (UniqueName: \"kubernetes.io/projected/fd743e06-bc6b-4891-97c9-b5eed07b14a9-kube-api-access-x7ghb\") pod \"fd743e06-bc6b-4891-97c9-b5eed07b14a9\" (UID: \"fd743e06-bc6b-4891-97c9-b5eed07b14a9\") "
Jul 06 18:58:22.989082 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:22.989060    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fd743e06-bc6b-4891-97c9-b5eed07b14a9-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "fd743e06-bc6b-4891-97c9-b5eed07b14a9" (UID: "fd743e06-bc6b-4891-97c9-b5eed07b14a9"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 18:58:22.989155 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:22.989138    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fd743e06-bc6b-4891-97c9-b5eed07b14a9-openshift-service-ca\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:58:22.990863 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:22.990833    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fd743e06-bc6b-4891-97c9-b5eed07b14a9-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "fd743e06-bc6b-4891-97c9-b5eed07b14a9" (UID: "fd743e06-bc6b-4891-97c9-b5eed07b14a9"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 18:58:22.990977 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:22.990922    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fd743e06-bc6b-4891-97c9-b5eed07b14a9-kube-api-access-x7ghb" (OuterVolumeSpecName: "kube-api-access-x7ghb") pod "fd743e06-bc6b-4891-97c9-b5eed07b14a9" (UID: "fd743e06-bc6b-4891-97c9-b5eed07b14a9"). InnerVolumeSpecName "kube-api-access-x7ghb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 18:58:23.089615 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:23.089576    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fd743e06-bc6b-4891-97c9-b5eed07b14a9-maas-api-tls\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:58:23.089615 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:23.089608    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x7ghb\" (UniqueName: \"kubernetes.io/projected/fd743e06-bc6b-4891-97c9-b5eed07b14a9-kube-api-access-x7ghb\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:58:23.754108 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:23.754003    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-b67177-6dff4d8d49-zd4qr"
Jul 06 18:58:23.754108 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:23.754042    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-b67177-6dff4d8d49-zd4qr" event={"ID":"fd743e06-bc6b-4891-97c9-b5eed07b14a9","Type":"ContainerDied","Data":"8585b176b6e227eaf78073d48c89969e34d99765349ea75b60b4a936faea5074"}
Jul 06 18:58:23.754108 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:23.754103    2580 scope.go:117] "RemoveContainer" containerID="ddb047d13d864a0c84f21d8b9e05fe82454eb5b8f1a8e61b3852d2c92e53bcf5"
Jul 06 18:58:23.772485 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:23.772456    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-b67177-6dff4d8d49-zd4qr"]
Jul 06 18:58:23.776669 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:23.776645    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-b67177-6dff4d8d49-zd4qr"]
Jul 06 18:58:25.323701 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:25.323669    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fd743e06-bc6b-4891-97c9-b5eed07b14a9" path="/var/lib/kubelet/pods/fd743e06-bc6b-4891-97c9-b5eed07b14a9/volumes"
Jul 06 18:58:26.636654 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:26.636627    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-8c615a-openshift-default-5b86f7bbcf-kkrr8"
Jul 06 18:58:26.725491 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:26.725394    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fef133e7-c5e3-4fa3-a6a5-f57239b3b56e-istio-token\") pod \"fef133e7-c5e3-4fa3-a6a5-f57239b3b56e\" (UID: \"fef133e7-c5e3-4fa3-a6a5-f57239b3b56e\") "
Jul 06 18:58:26.725491 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:26.725440    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fef133e7-c5e3-4fa3-a6a5-f57239b3b56e-workload-socket\") pod \"fef133e7-c5e3-4fa3-a6a5-f57239b3b56e\" (UID: \"fef133e7-c5e3-4fa3-a6a5-f57239b3b56e\") "
Jul 06 18:58:26.725491 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:26.725467    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fef133e7-c5e3-4fa3-a6a5-f57239b3b56e-istio-envoy\") pod \"fef133e7-c5e3-4fa3-a6a5-f57239b3b56e\" (UID: \"fef133e7-c5e3-4fa3-a6a5-f57239b3b56e\") "
Jul 06 18:58:26.725491 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:26.725485    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fef133e7-c5e3-4fa3-a6a5-f57239b3b56e-istiod-ca-cert\") pod \"fef133e7-c5e3-4fa3-a6a5-f57239b3b56e\" (UID: \"fef133e7-c5e3-4fa3-a6a5-f57239b3b56e\") "
Jul 06 18:58:26.725811 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:26.725521    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fef133e7-c5e3-4fa3-a6a5-f57239b3b56e-workload-certs\") pod \"fef133e7-c5e3-4fa3-a6a5-f57239b3b56e\" (UID: \"fef133e7-c5e3-4fa3-a6a5-f57239b3b56e\") "
Jul 06 18:58:26.725811 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:26.725551    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fef133e7-c5e3-4fa3-a6a5-f57239b3b56e-istio-podinfo\") pod \"fef133e7-c5e3-4fa3-a6a5-f57239b3b56e\" (UID: \"fef133e7-c5e3-4fa3-a6a5-f57239b3b56e\") "
Jul 06 18:58:26.725811 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:26.725577    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kjjhk\" (UniqueName: \"kubernetes.io/projected/fef133e7-c5e3-4fa3-a6a5-f57239b3b56e-kube-api-access-kjjhk\") pod \"fef133e7-c5e3-4fa3-a6a5-f57239b3b56e\" (UID: \"fef133e7-c5e3-4fa3-a6a5-f57239b3b56e\") "
Jul 06 18:58:26.725811 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:26.725632    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fef133e7-c5e3-4fa3-a6a5-f57239b3b56e-istio-data\") pod \"fef133e7-c5e3-4fa3-a6a5-f57239b3b56e\" (UID: \"fef133e7-c5e3-4fa3-a6a5-f57239b3b56e\") "
Jul 06 18:58:26.726067 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:26.725831    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fef133e7-c5e3-4fa3-a6a5-f57239b3b56e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "fef133e7-c5e3-4fa3-a6a5-f57239b3b56e" (UID: "fef133e7-c5e3-4fa3-a6a5-f57239b3b56e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 18:58:26.726067 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:26.725893    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fef133e7-c5e3-4fa3-a6a5-f57239b3b56e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "fef133e7-c5e3-4fa3-a6a5-f57239b3b56e" (UID: "fef133e7-c5e3-4fa3-a6a5-f57239b3b56e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 18:58:26.726067 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:26.725936    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fef133e7-c5e3-4fa3-a6a5-f57239b3b56e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "fef133e7-c5e3-4fa3-a6a5-f57239b3b56e" (UID: "fef133e7-c5e3-4fa3-a6a5-f57239b3b56e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 18:58:26.726701 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:26.726679    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fef133e7-c5e3-4fa3-a6a5-f57239b3b56e-istio-data" (OuterVolumeSpecName: "istio-data") pod "fef133e7-c5e3-4fa3-a6a5-f57239b3b56e" (UID: "fef133e7-c5e3-4fa3-a6a5-f57239b3b56e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 18:58:26.727694 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:26.727675    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/fef133e7-c5e3-4fa3-a6a5-f57239b3b56e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "fef133e7-c5e3-4fa3-a6a5-f57239b3b56e" (UID: "fef133e7-c5e3-4fa3-a6a5-f57239b3b56e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 06 18:58:26.727800 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:26.727772    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fef133e7-c5e3-4fa3-a6a5-f57239b3b56e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "fef133e7-c5e3-4fa3-a6a5-f57239b3b56e" (UID: "fef133e7-c5e3-4fa3-a6a5-f57239b3b56e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 18:58:26.728029 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:26.727986    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fef133e7-c5e3-4fa3-a6a5-f57239b3b56e-istio-token" (OuterVolumeSpecName: "istio-token") pod "fef133e7-c5e3-4fa3-a6a5-f57239b3b56e" (UID: "fef133e7-c5e3-4fa3-a6a5-f57239b3b56e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 18:58:26.728122 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:26.727994    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fef133e7-c5e3-4fa3-a6a5-f57239b3b56e-kube-api-access-kjjhk" (OuterVolumeSpecName: "kube-api-access-kjjhk") pod "fef133e7-c5e3-4fa3-a6a5-f57239b3b56e" (UID: "fef133e7-c5e3-4fa3-a6a5-f57239b3b56e"). InnerVolumeSpecName "kube-api-access-kjjhk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 18:58:26.772085 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:26.772036    2580 generic.go:356] "Generic (PLEG): container finished" podID="fef133e7-c5e3-4fa3-a6a5-f57239b3b56e" containerID="0bea27fa04868846d09f5609b681bd6c270d092c56f06e207c9984064a59a990" exitCode=0
Jul 06 18:58:26.772288 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:26.772092    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-8c615a-openshift-default-5b86f7bbcf-kkrr8"
Jul 06 18:58:26.772288 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:26.772099    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-8c615a-openshift-default-5b86f7bbcf-kkrr8" event={"ID":"fef133e7-c5e3-4fa3-a6a5-f57239b3b56e","Type":"ContainerDied","Data":"0bea27fa04868846d09f5609b681bd6c270d092c56f06e207c9984064a59a990"}
Jul 06 18:58:26.772288 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:26.772163    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-8c615a-openshift-default-5b86f7bbcf-kkrr8" event={"ID":"fef133e7-c5e3-4fa3-a6a5-f57239b3b56e","Type":"ContainerDied","Data":"8b9f644ff1c34c41d302c5700bf3658b186da8835634c2a1bd98fbc3080bb2a2"}
Jul 06 18:58:26.772288 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:26.772192    2580 scope.go:117] "RemoveContainer" containerID="0bea27fa04868846d09f5609b681bd6c270d092c56f06e207c9984064a59a990"
Jul 06 18:58:26.781164 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:26.781145    2580 scope.go:117] "RemoveContainer" containerID="97616b491f19f13001a9a37cfb0cb797e1dd901959eeaea648c4cc903f160052"
Jul 06 18:58:26.789053 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:26.789034    2580 scope.go:117] "RemoveContainer" containerID="0bea27fa04868846d09f5609b681bd6c270d092c56f06e207c9984064a59a990"
Jul 06 18:58:26.789325 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:58:26.789302    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0bea27fa04868846d09f5609b681bd6c270d092c56f06e207c9984064a59a990\": container with ID starting with 0bea27fa04868846d09f5609b681bd6c270d092c56f06e207c9984064a59a990 not found: ID does not exist" containerID="0bea27fa04868846d09f5609b681bd6c270d092c56f06e207c9984064a59a990"
Jul 06 18:58:26.789412 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:26.789340    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0bea27fa04868846d09f5609b681bd6c270d092c56f06e207c9984064a59a990"} err="failed to get container status \"0bea27fa04868846d09f5609b681bd6c270d092c56f06e207c9984064a59a990\": rpc error: code = NotFound desc = could not find container \"0bea27fa04868846d09f5609b681bd6c270d092c56f06e207c9984064a59a990\": container with ID starting with 0bea27fa04868846d09f5609b681bd6c270d092c56f06e207c9984064a59a990 not found: ID does not exist"
Jul 06 18:58:26.789412 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:26.789364    2580 scope.go:117] "RemoveContainer" containerID="97616b491f19f13001a9a37cfb0cb797e1dd901959eeaea648c4cc903f160052"
Jul 06 18:58:26.789624 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:58:26.789605    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"97616b491f19f13001a9a37cfb0cb797e1dd901959eeaea648c4cc903f160052\": container with ID starting with 97616b491f19f13001a9a37cfb0cb797e1dd901959eeaea648c4cc903f160052 not found: ID does not exist" containerID="97616b491f19f13001a9a37cfb0cb797e1dd901959eeaea648c4cc903f160052"
Jul 06 18:58:26.789667 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:26.789631    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"97616b491f19f13001a9a37cfb0cb797e1dd901959eeaea648c4cc903f160052"} err="failed to get container status \"97616b491f19f13001a9a37cfb0cb797e1dd901959eeaea648c4cc903f160052\": rpc error: code = NotFound desc = could not find container \"97616b491f19f13001a9a37cfb0cb797e1dd901959eeaea648c4cc903f160052\": container with ID starting with 97616b491f19f13001a9a37cfb0cb797e1dd901959eeaea648c4cc903f160052 not found: ID does not exist"
Jul 06 18:58:26.826834 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:26.826799    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fef133e7-c5e3-4fa3-a6a5-f57239b3b56e-credential-socket\") pod \"fef133e7-c5e3-4fa3-a6a5-f57239b3b56e\" (UID: \"fef133e7-c5e3-4fa3-a6a5-f57239b3b56e\") "
Jul 06 18:58:26.827000 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:26.826987    2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fef133e7-c5e3-4fa3-a6a5-f57239b3b56e-istio-podinfo\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:58:26.827000 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:26.826998    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kjjhk\" (UniqueName: \"kubernetes.io/projected/fef133e7-c5e3-4fa3-a6a5-f57239b3b56e-kube-api-access-kjjhk\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:58:26.827171 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:26.827007    2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fef133e7-c5e3-4fa3-a6a5-f57239b3b56e-istio-data\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:58:26.827171 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:26.827034    2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fef133e7-c5e3-4fa3-a6a5-f57239b3b56e-istio-token\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:58:26.827171 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:26.827043    2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fef133e7-c5e3-4fa3-a6a5-f57239b3b56e-workload-socket\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:58:26.827171 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:26.827054    2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fef133e7-c5e3-4fa3-a6a5-f57239b3b56e-istio-envoy\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:58:26.827171 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:26.827065    2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fef133e7-c5e3-4fa3-a6a5-f57239b3b56e-istiod-ca-cert\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:58:26.827171 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:26.827073    2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fef133e7-c5e3-4fa3-a6a5-f57239b3b56e-workload-certs\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:58:26.827171 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:26.827141    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fef133e7-c5e3-4fa3-a6a5-f57239b3b56e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "fef133e7-c5e3-4fa3-a6a5-f57239b3b56e" (UID: "fef133e7-c5e3-4fa3-a6a5-f57239b3b56e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 18:58:26.927862 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:26.927826    2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fef133e7-c5e3-4fa3-a6a5-f57239b3b56e-credential-socket\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:58:27.095964 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:27.095922    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-8c615a-openshift-default-5b86f7bbcf-kkrr8"]
Jul 06 18:58:27.100446 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:27.100418    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-8c615a-openshift-default-5b86f7bbcf-kkrr8"]
Jul 06 18:58:27.323048 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:27.322995    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fef133e7-c5e3-4fa3-a6a5-f57239b3b56e" path="/var/lib/kubelet/pods/fef133e7-c5e3-4fa3-a6a5-f57239b3b56e/volumes"
Jul 06 18:58:28.720469 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:28.720431    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-b67177-openshift-default-78b7697575-tz649"]
Jul 06 18:58:28.720883 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:28.720656    2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-b67177-openshift-default-78b7697575-tz649" podUID="782ee0c9-e518-4f88-a42d-73f6fc752038" containerName="istio-proxy" containerID="cri-o://b1dfd3541e60c277d482c9e051f20430d3696819213ca9e1c4f116617a916475" gracePeriod=30
Jul 06 18:58:29.384660 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:29.384627    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-5cb897bc47-8zbh8_b2278607-0062-4bc4-8f3f-57cc9a60f2a4/authorino/0.log"
Jul 06 18:58:32.290100 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:32.290059    2580 patch_prober.go:28] interesting pod/e2e-shared-b-b67177-openshift-default-78b7697575-tz649 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 06 18:58:32.290551 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:32.290119    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-b67177-openshift-default-78b7697575-tz649" podUID="782ee0c9-e518-4f88-a42d-73f6fc752038" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 06 18:58:33.885185 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:33.885152    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-c977q_dfe46fe3-2f4d-4855-97c1-38e4e974c6f8/manager/0.log"
Jul 06 18:58:33.980728 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:33.980700    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-b67177-openshift-default-78b7697575-tz649"
Jul 06 18:58:34.009669 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:34.009597    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-4vm88_e9cabd49-cdd7-4882-afc0-da156d82c239/manager/0.log"
Jul 06 18:58:34.087690 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:34.087654    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/782ee0c9-e518-4f88-a42d-73f6fc752038-workload-socket\") pod \"782ee0c9-e518-4f88-a42d-73f6fc752038\" (UID: \"782ee0c9-e518-4f88-a42d-73f6fc752038\") "
Jul 06 18:58:34.087690 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:34.087690    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p75rm\" (UniqueName: \"kubernetes.io/projected/782ee0c9-e518-4f88-a42d-73f6fc752038-kube-api-access-p75rm\") pod \"782ee0c9-e518-4f88-a42d-73f6fc752038\" (UID: \"782ee0c9-e518-4f88-a42d-73f6fc752038\") "
Jul 06 18:58:34.087947 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:34.087725    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/782ee0c9-e518-4f88-a42d-73f6fc752038-istio-podinfo\") pod \"782ee0c9-e518-4f88-a42d-73f6fc752038\" (UID: \"782ee0c9-e518-4f88-a42d-73f6fc752038\") "
Jul 06 18:58:34.087947 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:34.087747    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/782ee0c9-e518-4f88-a42d-73f6fc752038-credential-socket\") pod \"782ee0c9-e518-4f88-a42d-73f6fc752038\" (UID: \"782ee0c9-e518-4f88-a42d-73f6fc752038\") "
Jul 06 18:58:34.087947 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:34.087799    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/782ee0c9-e518-4f88-a42d-73f6fc752038-istiod-ca-cert\") pod \"782ee0c9-e518-4f88-a42d-73f6fc752038\" (UID: \"782ee0c9-e518-4f88-a42d-73f6fc752038\") "
Jul 06 18:58:34.087947 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:34.087841    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/782ee0c9-e518-4f88-a42d-73f6fc752038-istio-data\") pod \"782ee0c9-e518-4f88-a42d-73f6fc752038\" (UID: \"782ee0c9-e518-4f88-a42d-73f6fc752038\") "
Jul 06 18:58:34.087947 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:34.087863    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/782ee0c9-e518-4f88-a42d-73f6fc752038-istio-token\") pod \"782ee0c9-e518-4f88-a42d-73f6fc752038\" (UID: \"782ee0c9-e518-4f88-a42d-73f6fc752038\") "
Jul 06 18:58:34.087947 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:34.087947    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/782ee0c9-e518-4f88-a42d-73f6fc752038-istio-envoy\") pod \"782ee0c9-e518-4f88-a42d-73f6fc752038\" (UID: \"782ee0c9-e518-4f88-a42d-73f6fc752038\") "
Jul 06 18:58:34.088312 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:34.087981    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/782ee0c9-e518-4f88-a42d-73f6fc752038-workload-certs\") pod \"782ee0c9-e518-4f88-a42d-73f6fc752038\" (UID: \"782ee0c9-e518-4f88-a42d-73f6fc752038\") "
Jul 06 18:58:34.088312 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:34.087995    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/782ee0c9-e518-4f88-a42d-73f6fc752038-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "782ee0c9-e518-4f88-a42d-73f6fc752038" (UID: "782ee0c9-e518-4f88-a42d-73f6fc752038"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 18:58:34.088312 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:34.088153    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/782ee0c9-e518-4f88-a42d-73f6fc752038-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "782ee0c9-e518-4f88-a42d-73f6fc752038" (UID: "782ee0c9-e518-4f88-a42d-73f6fc752038"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 18:58:34.088312 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:34.088301    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/782ee0c9-e518-4f88-a42d-73f6fc752038-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "782ee0c9-e518-4f88-a42d-73f6fc752038" (UID: "782ee0c9-e518-4f88-a42d-73f6fc752038"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 18:58:34.088537 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:34.088315    2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/782ee0c9-e518-4f88-a42d-73f6fc752038-workload-socket\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:58:34.088537 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:34.088335    2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/782ee0c9-e518-4f88-a42d-73f6fc752038-credential-socket\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:58:34.088646 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:34.088599    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/782ee0c9-e518-4f88-a42d-73f6fc752038-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "782ee0c9-e518-4f88-a42d-73f6fc752038" (UID: "782ee0c9-e518-4f88-a42d-73f6fc752038"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 18:58:34.088855 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:34.088828    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/782ee0c9-e518-4f88-a42d-73f6fc752038-istio-data" (OuterVolumeSpecName: "istio-data") pod "782ee0c9-e518-4f88-a42d-73f6fc752038" (UID: "782ee0c9-e518-4f88-a42d-73f6fc752038"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 18:58:34.090160 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:34.090134    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/782ee0c9-e518-4f88-a42d-73f6fc752038-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "782ee0c9-e518-4f88-a42d-73f6fc752038" (UID: "782ee0c9-e518-4f88-a42d-73f6fc752038"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 18:58:34.090397 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:34.090367    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/782ee0c9-e518-4f88-a42d-73f6fc752038-kube-api-access-p75rm" (OuterVolumeSpecName: "kube-api-access-p75rm") pod "782ee0c9-e518-4f88-a42d-73f6fc752038" (UID: "782ee0c9-e518-4f88-a42d-73f6fc752038"). InnerVolumeSpecName "kube-api-access-p75rm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 18:58:34.090481 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:34.090419    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/782ee0c9-e518-4f88-a42d-73f6fc752038-istio-token" (OuterVolumeSpecName: "istio-token") pod "782ee0c9-e518-4f88-a42d-73f6fc752038" (UID: "782ee0c9-e518-4f88-a42d-73f6fc752038"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 18:58:34.090481 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:34.090434    2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/782ee0c9-e518-4f88-a42d-73f6fc752038-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "782ee0c9-e518-4f88-a42d-73f6fc752038" (UID: "782ee0c9-e518-4f88-a42d-73f6fc752038"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 06 18:58:34.131854 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:34.131809    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-79c74f6bbb-cbdff_8ba23aff-6404-4870-8919-6dc1ca89fa0c/maas-api/0.log"
Jul 06 18:58:34.189717 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:34.189669    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p75rm\" (UniqueName: \"kubernetes.io/projected/782ee0c9-e518-4f88-a42d-73f6fc752038-kube-api-access-p75rm\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:58:34.189717 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:34.189716    2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/782ee0c9-e518-4f88-a42d-73f6fc752038-istio-podinfo\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:58:34.189717 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:34.189726    2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/782ee0c9-e518-4f88-a42d-73f6fc752038-istiod-ca-cert\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:58:34.189913 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:34.189739    2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/782ee0c9-e518-4f88-a42d-73f6fc752038-istio-data\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:58:34.189913 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:34.189746    2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/782ee0c9-e518-4f88-a42d-73f6fc752038-istio-token\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:58:34.189913 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:34.189754    2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/782ee0c9-e518-4f88-a42d-73f6fc752038-istio-envoy\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:58:34.189913 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:34.189762    2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/782ee0c9-e518-4f88-a42d-73f6fc752038-workload-certs\") on node \"ip-10-0-134-113.ec2.internal\" DevicePath \"\""
Jul 06 18:58:34.236916 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:34.236885    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29722710-ldj78_44be2b8f-8aac-4423-aeea-03c7d2caea25/cleanup/0.log"
Jul 06 18:58:34.346853 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:34.346827    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29722725-dx9b7_17eeefb8-1ca4-4c82-b73f-fe031dbbd201/cleanup/0.log"
Jul 06 18:58:34.464744 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:34.464714    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-c84d79b86-pc4x6_ab01db64-3934-426a-a989-760248f8c0cd/manager/0.log"
Jul 06 18:58:34.574925 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:34.574897    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-rt527_a116c1ff-d114-4ba8-8a6d-eec4317c6978/server/0.log"
Jul 06 18:58:34.684401 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:34.684333    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-cjcm8_b77b21f6-57a3-476e-96c0-f9c87a61683e/manager/0.log"
Jul 06 18:58:34.812357 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:34.812323    2580 generic.go:356] "Generic (PLEG): container finished" podID="782ee0c9-e518-4f88-a42d-73f6fc752038" containerID="b1dfd3541e60c277d482c9e051f20430d3696819213ca9e1c4f116617a916475" exitCode=0
Jul 06 18:58:34.812517 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:34.812396    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-b67177-openshift-default-78b7697575-tz649"
Jul 06 18:58:34.812517 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:34.812405    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-b67177-openshift-default-78b7697575-tz649" event={"ID":"782ee0c9-e518-4f88-a42d-73f6fc752038","Type":"ContainerDied","Data":"b1dfd3541e60c277d482c9e051f20430d3696819213ca9e1c4f116617a916475"}
Jul 06 18:58:34.812517 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:34.812466    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-b67177-openshift-default-78b7697575-tz649" event={"ID":"782ee0c9-e518-4f88-a42d-73f6fc752038","Type":"ContainerDied","Data":"fbe9b49175eabf52460393fc14df2c584955c522bed84977a0b5bf0d02e2e6ae"}
Jul 06 18:58:34.813481 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:34.812496    2580 scope.go:117] "RemoveContainer" containerID="b1dfd3541e60c277d482c9e051f20430d3696819213ca9e1c4f116617a916475"
Jul 06 18:58:34.826951 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:34.826913    2580 scope.go:117] "RemoveContainer" containerID="01ad046110af22074d68cebe3656a28cb6077d6b8c1cde8003097c46ee60818a"
Jul 06 18:58:34.834599 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:34.834583    2580 scope.go:117] "RemoveContainer" containerID="b1dfd3541e60c277d482c9e051f20430d3696819213ca9e1c4f116617a916475"
Jul 06 18:58:34.834844 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:58:34.834817    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b1dfd3541e60c277d482c9e051f20430d3696819213ca9e1c4f116617a916475\": container with ID starting with b1dfd3541e60c277d482c9e051f20430d3696819213ca9e1c4f116617a916475 not found: ID does not exist" containerID="b1dfd3541e60c277d482c9e051f20430d3696819213ca9e1c4f116617a916475"
Jul 06 18:58:34.834885 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:34.834858    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b1dfd3541e60c277d482c9e051f20430d3696819213ca9e1c4f116617a916475"} err="failed to get container status \"b1dfd3541e60c277d482c9e051f20430d3696819213ca9e1c4f116617a916475\": rpc error: code = NotFound desc = could not find container \"b1dfd3541e60c277d482c9e051f20430d3696819213ca9e1c4f116617a916475\": container with ID starting with b1dfd3541e60c277d482c9e051f20430d3696819213ca9e1c4f116617a916475 not found: ID does not exist"
Jul 06 18:58:34.834885 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:34.834878    2580 scope.go:117] "RemoveContainer" containerID="01ad046110af22074d68cebe3656a28cb6077d6b8c1cde8003097c46ee60818a"
Jul 06 18:58:34.835129 ip-10-0-134-113 kubenswrapper[2580]: E0706 18:58:34.835112    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"01ad046110af22074d68cebe3656a28cb6077d6b8c1cde8003097c46ee60818a\": container with ID starting with 01ad046110af22074d68cebe3656a28cb6077d6b8c1cde8003097c46ee60818a not found: ID does not exist" containerID="01ad046110af22074d68cebe3656a28cb6077d6b8c1cde8003097c46ee60818a"
Jul 06 18:58:34.835196 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:34.835136    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"01ad046110af22074d68cebe3656a28cb6077d6b8c1cde8003097c46ee60818a"} err="failed to get container status \"01ad046110af22074d68cebe3656a28cb6077d6b8c1cde8003097c46ee60818a\": rpc error: code = NotFound desc = could not find container \"01ad046110af22074d68cebe3656a28cb6077d6b8c1cde8003097c46ee60818a\": container with ID starting with 01ad046110af22074d68cebe3656a28cb6077d6b8c1cde8003097c46ee60818a not found: ID does not exist"
Jul 06 18:58:34.838232 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:34.838212    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-b67177-openshift-default-78b7697575-tz649"]
Jul 06 18:58:34.842282 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:34.842262    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-b67177-openshift-default-78b7697575-tz649"]
Jul 06 18:58:35.061327 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:35.061253    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6bfdf69c66-tbm2m_6bc2ec3e-8ccd-41a1-a9d8-6013a0535dfb/copy-manifests/0.log"
Jul 06 18:58:35.071103 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:35.071082    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6bfdf69c66-tbm2m_6bc2ec3e-8ccd-41a1-a9d8-6013a0535dfb/manager/0.log"
Jul 06 18:58:35.180176 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:35.180149    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-69fb4594cf-859xw_c481500b-94ce-4b34-8a62-755f86931529/postgres/0.log"
Jul 06 18:58:35.324627 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:35.324598    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="782ee0c9-e518-4f88-a42d-73f6fc752038" path="/var/lib/kubelet/pods/782ee0c9-e518-4f88-a42d-73f6fc752038/volumes"
Jul 06 18:58:36.489899 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:36.489871    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-5cb897bc47-8zbh8_b2278607-0062-4bc4-8f3f-57cc9a60f2a4/authorino/0.log"
Jul 06 18:58:36.611473 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:36.611444    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-l47mq_5a1328d5-22f7-4276-a5b5-506f97578629/manager/0.log"
Jul 06 18:58:36.725801 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:36.725771    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-px77q_d8f47480-21bb-4696-b306-c0b5099e9f0e/manager/0.log"
Jul 06 18:58:36.953677 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:36.953645    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-xm6rr_37a41edd-6d01-4bdc-b374-9cb0ed5a23ac/registry-server/0.log"
Jul 06 18:58:37.318556 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:37.318472    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-hdrzx_5f34ac69-ec56-4482-a77a-d60caac43ac0/manager/0.log"
Jul 06 18:58:37.673073 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:37.673043    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4m2pkz_5d7cc344-17c6-4259-90cf-4422e436112b/istio-proxy/0.log"
Jul 06 18:58:38.012451 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:38.012371    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-68bf6db468-pvqn9_6f86a0d6-2cde-4e56-8348-9ad3351f9500/kube-auth-proxy/0.log"
Jul 06 18:58:38.129029 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:38.128998    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-tlmbg_75b7f81b-fdb8-4d09-8317-8c9323960ed5/istio-proxy/1.log"
Jul 06 18:58:38.248801 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:38.248767    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-59ldh_e46693cd-0f3e-4ed5-a5f5-748daad00f43/payload-pre-processing/0.log"
Jul 06 18:58:38.828717 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:38.828687    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-smh9r_50799c4d-2fd5-41a5-aa2b-cc1eb776ae9e/main/0.log"
Jul 06 18:58:43.518918 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:43.518883    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-8xw5b/must-gather-5bwfj"]
Jul 06 18:58:43.519310 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:43.519274    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fef133e7-c5e3-4fa3-a6a5-f57239b3b56e" containerName="istio-proxy"
Jul 06 18:58:43.519310 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:43.519287    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="fef133e7-c5e3-4fa3-a6a5-f57239b3b56e" containerName="istio-proxy"
Jul 06 18:58:43.519310 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:43.519298    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="782ee0c9-e518-4f88-a42d-73f6fc752038" containerName="istio-proxy"
Jul 06 18:58:43.519310 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:43.519305    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="782ee0c9-e518-4f88-a42d-73f6fc752038" containerName="istio-proxy"
Jul 06 18:58:43.519440 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:43.519313    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8dad6f0a-b625-4617-a3d4-b55615c16401" containerName="main"
Jul 06 18:58:43.519440 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:43.519319    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="8dad6f0a-b625-4617-a3d4-b55615c16401" containerName="main"
Jul 06 18:58:43.519440 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:43.519329    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="455f11a6-7dde-4dff-8aab-9faad1d6a318" containerName="maas-api"
Jul 06 18:58:43.519440 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:43.519338    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="455f11a6-7dde-4dff-8aab-9faad1d6a318" containerName="maas-api"
Jul 06 18:58:43.519440 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:43.519348    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="782ee0c9-e518-4f88-a42d-73f6fc752038" containerName="istio-proxy"
Jul 06 18:58:43.519440 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:43.519353    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="782ee0c9-e518-4f88-a42d-73f6fc752038" containerName="istio-proxy"
Jul 06 18:58:43.519440 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:43.519363    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fef133e7-c5e3-4fa3-a6a5-f57239b3b56e" containerName="istio-proxy"
Jul 06 18:58:43.519440 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:43.519368    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="fef133e7-c5e3-4fa3-a6a5-f57239b3b56e" containerName="istio-proxy"
Jul 06 18:58:43.519440 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:43.519378    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fd743e06-bc6b-4891-97c9-b5eed07b14a9" containerName="maas-api"
Jul 06 18:58:43.519440 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:43.519384    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="fd743e06-bc6b-4891-97c9-b5eed07b14a9" containerName="maas-api"
Jul 06 18:58:43.519744 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:43.519446    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="fd743e06-bc6b-4891-97c9-b5eed07b14a9" containerName="maas-api"
Jul 06 18:58:43.519744 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:43.519454    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="782ee0c9-e518-4f88-a42d-73f6fc752038" containerName="istio-proxy"
Jul 06 18:58:43.519744 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:43.519461    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="fef133e7-c5e3-4fa3-a6a5-f57239b3b56e" containerName="istio-proxy"
Jul 06 18:58:43.519744 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:43.519467    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="fef133e7-c5e3-4fa3-a6a5-f57239b3b56e" containerName="istio-proxy"
Jul 06 18:58:43.519744 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:43.519474    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="455f11a6-7dde-4dff-8aab-9faad1d6a318" containerName="maas-api"
Jul 06 18:58:43.519744 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:43.519479    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="8dad6f0a-b625-4617-a3d4-b55615c16401" containerName="main"
Jul 06 18:58:43.519744 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:43.519591    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="782ee0c9-e518-4f88-a42d-73f6fc752038" containerName="istio-proxy"
Jul 06 18:58:43.523921 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:43.523901    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-8xw5b/must-gather-5bwfj"
Jul 06 18:58:43.527667 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:43.527645    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-8xw5b\"/\"openshift-service-ca.crt\""
Jul 06 18:58:43.527790 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:43.527779    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-8xw5b\"/\"kube-root-ca.crt\""
Jul 06 18:58:43.529062 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:43.529043    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-8xw5b\"/\"default-dockercfg-xdgdk\""
Jul 06 18:58:43.533333 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:43.533309    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-8xw5b/must-gather-5bwfj"]
Jul 06 18:58:43.572981 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:43.572950    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/91411e03-22cc-454f-bff0-b1ffcf4d102e-must-gather-output\") pod \"must-gather-5bwfj\" (UID: \"91411e03-22cc-454f-bff0-b1ffcf4d102e\") " pod="openshift-must-gather-8xw5b/must-gather-5bwfj"
Jul 06 18:58:43.573186 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:43.572991    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2h6h7\" (UniqueName: \"kubernetes.io/projected/91411e03-22cc-454f-bff0-b1ffcf4d102e-kube-api-access-2h6h7\") pod \"must-gather-5bwfj\" (UID: \"91411e03-22cc-454f-bff0-b1ffcf4d102e\") " pod="openshift-must-gather-8xw5b/must-gather-5bwfj"
Jul 06 18:58:43.673802 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:43.673762    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/91411e03-22cc-454f-bff0-b1ffcf4d102e-must-gather-output\") pod \"must-gather-5bwfj\" (UID: \"91411e03-22cc-454f-bff0-b1ffcf4d102e\") " pod="openshift-must-gather-8xw5b/must-gather-5bwfj"
Jul 06 18:58:43.674050 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:43.673811    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2h6h7\" (UniqueName: \"kubernetes.io/projected/91411e03-22cc-454f-bff0-b1ffcf4d102e-kube-api-access-2h6h7\") pod \"must-gather-5bwfj\" (UID: \"91411e03-22cc-454f-bff0-b1ffcf4d102e\") " pod="openshift-must-gather-8xw5b/must-gather-5bwfj"
Jul 06 18:58:43.674130 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:43.674112    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/91411e03-22cc-454f-bff0-b1ffcf4d102e-must-gather-output\") pod \"must-gather-5bwfj\" (UID: \"91411e03-22cc-454f-bff0-b1ffcf4d102e\") " pod="openshift-must-gather-8xw5b/must-gather-5bwfj"
Jul 06 18:58:43.686958 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:43.686939    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2h6h7\" (UniqueName: \"kubernetes.io/projected/91411e03-22cc-454f-bff0-b1ffcf4d102e-kube-api-access-2h6h7\") pod \"must-gather-5bwfj\" (UID: \"91411e03-22cc-454f-bff0-b1ffcf4d102e\") " pod="openshift-must-gather-8xw5b/must-gather-5bwfj"
Jul 06 18:58:43.833319 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:43.833292    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-8xw5b/must-gather-5bwfj"
Jul 06 18:58:43.968719 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:43.968694    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-8xw5b/must-gather-5bwfj"]
Jul 06 18:58:43.970506 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:58:43.970472    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod91411e03_22cc_454f_bff0_b1ffcf4d102e.slice/crio-020a9e18f75e76328d1ef6a5690192577a6ca8d630082bb1064ed334ab63ba2d WatchSource:0}: Error finding container 020a9e18f75e76328d1ef6a5690192577a6ca8d630082bb1064ed334ab63ba2d: Status 404 returned error can't find the container with id 020a9e18f75e76328d1ef6a5690192577a6ca8d630082bb1064ed334ab63ba2d
Jul 06 18:58:43.972279 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:43.972262    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 06 18:58:44.862310 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:44.862279    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-8xw5b/must-gather-5bwfj" event={"ID":"91411e03-22cc-454f-bff0-b1ffcf4d102e","Type":"ContainerStarted","Data":"f6661971beae58d556d424f477183a4a3e125954aee21cb57916c84a04f17ff8"}
Jul 06 18:58:44.862709 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:44.862324    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-8xw5b/must-gather-5bwfj" event={"ID":"91411e03-22cc-454f-bff0-b1ffcf4d102e","Type":"ContainerStarted","Data":"020a9e18f75e76328d1ef6a5690192577a6ca8d630082bb1064ed334ab63ba2d"}
Jul 06 18:58:45.871905 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:45.871074    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-8xw5b/must-gather-5bwfj" event={"ID":"91411e03-22cc-454f-bff0-b1ffcf4d102e","Type":"ContainerStarted","Data":"7ae72a4344af9f8013f0c89eaa615537ba37f7543b3dfbaaa169b7b434aa659b"}
Jul 06 18:58:45.891924 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:45.891866    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-8xw5b/must-gather-5bwfj" podStartSLOduration=2.096506902 podStartE2EDuration="2.891846346s" totalImagesPullingTime="795.339444ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:58:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:58:45.889383849 +0000 UTC m=+2461.046782336" watchObservedRunningTime="2026-07-06 18:58:45.891846346 +0000 UTC m=+2461.049244833"
Jul 06 18:58:46.439991 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:46.439950    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-xzqch_3b6b323e-eca7-4c68-87a7-0d48a9114a2c/global-pull-secret-syncer/0.log"
Jul 06 18:58:46.571962 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:46.571933    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-b7jm9_f117fdd5-8a61-484e-8ebc-668f89723813/konnectivity-agent/0.log"
Jul 06 18:58:46.696409 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:46.696327    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-134-113.ec2.internal_66d40220305fe03479bbe46175f2cd79/haproxy/0.log"
Jul 06 18:58:52.035156 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:52.035125    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-5cb897bc47-8zbh8_b2278607-0062-4bc4-8f3f-57cc9a60f2a4/authorino/0.log"
Jul 06 18:58:52.073493 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:52.073459    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-l47mq_5a1328d5-22f7-4276-a5b5-506f97578629/manager/0.log"
Jul 06 18:58:52.098959 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:52.098930    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-px77q_d8f47480-21bb-4696-b306-c0b5099e9f0e/manager/0.log"
Jul 06 18:58:52.179246 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:52.179205    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-xm6rr_37a41edd-6d01-4bdc-b374-9cb0ed5a23ac/registry-server/0.log"
Jul 06 18:58:52.473063 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:52.470785    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-hdrzx_5f34ac69-ec56-4482-a77a-d60caac43ac0/manager/0.log"
Jul 06 18:58:54.434646 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:54.434617    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-r5gqj_aed7c886-98ef-45f3-8b70-23a67b244fec/node-exporter/0.log"
Jul 06 18:58:54.453733 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:54.453693    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-r5gqj_aed7c886-98ef-45f3-8b70-23a67b244fec/kube-rbac-proxy/0.log"
Jul 06 18:58:54.477475 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:54.477449    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-r5gqj_aed7c886-98ef-45f3-8b70-23a67b244fec/init-textfile/0.log"
Jul 06 18:58:56.111529 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:56.111491    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-8xw5b/perf-node-gather-daemonset-6bq9d"]
Jul 06 18:58:56.116776 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:56.116749    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-8xw5b/perf-node-gather-daemonset-6bq9d"
Jul 06 18:58:56.126330 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:56.126146    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-8xw5b/perf-node-gather-daemonset-6bq9d"]
Jul 06 18:58:56.208989 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:56.208954    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q5fk7\" (UniqueName: \"kubernetes.io/projected/042ddd02-3a84-4075-8a71-bcbdd62b0108-kube-api-access-q5fk7\") pod \"perf-node-gather-daemonset-6bq9d\" (UID: \"042ddd02-3a84-4075-8a71-bcbdd62b0108\") " pod="openshift-must-gather-8xw5b/perf-node-gather-daemonset-6bq9d"
Jul 06 18:58:56.209255 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:56.209237    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/042ddd02-3a84-4075-8a71-bcbdd62b0108-proc\") pod \"perf-node-gather-daemonset-6bq9d\" (UID: \"042ddd02-3a84-4075-8a71-bcbdd62b0108\") " pod="openshift-must-gather-8xw5b/perf-node-gather-daemonset-6bq9d"
Jul 06 18:58:56.209415 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:56.209399    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/042ddd02-3a84-4075-8a71-bcbdd62b0108-lib-modules\") pod \"perf-node-gather-daemonset-6bq9d\" (UID: \"042ddd02-3a84-4075-8a71-bcbdd62b0108\") " pod="openshift-must-gather-8xw5b/perf-node-gather-daemonset-6bq9d"
Jul 06 18:58:56.209510 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:56.209496    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/042ddd02-3a84-4075-8a71-bcbdd62b0108-podres\") pod \"perf-node-gather-daemonset-6bq9d\" (UID: \"042ddd02-3a84-4075-8a71-bcbdd62b0108\") " pod="openshift-must-gather-8xw5b/perf-node-gather-daemonset-6bq9d"
Jul 06 18:58:56.209672 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:56.209659    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/042ddd02-3a84-4075-8a71-bcbdd62b0108-sys\") pod \"perf-node-gather-daemonset-6bq9d\" (UID: \"042ddd02-3a84-4075-8a71-bcbdd62b0108\") " pod="openshift-must-gather-8xw5b/perf-node-gather-daemonset-6bq9d"
Jul 06 18:58:56.311418 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:56.311366    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q5fk7\" (UniqueName: \"kubernetes.io/projected/042ddd02-3a84-4075-8a71-bcbdd62b0108-kube-api-access-q5fk7\") pod \"perf-node-gather-daemonset-6bq9d\" (UID: \"042ddd02-3a84-4075-8a71-bcbdd62b0108\") " pod="openshift-must-gather-8xw5b/perf-node-gather-daemonset-6bq9d"
Jul 06 18:58:56.311418 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:56.311423    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/042ddd02-3a84-4075-8a71-bcbdd62b0108-proc\") pod \"perf-node-gather-daemonset-6bq9d\" (UID: \"042ddd02-3a84-4075-8a71-bcbdd62b0108\") " pod="openshift-must-gather-8xw5b/perf-node-gather-daemonset-6bq9d"
Jul 06 18:58:56.311678 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:56.311498    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/042ddd02-3a84-4075-8a71-bcbdd62b0108-lib-modules\") pod \"perf-node-gather-daemonset-6bq9d\" (UID: \"042ddd02-3a84-4075-8a71-bcbdd62b0108\") " pod="openshift-must-gather-8xw5b/perf-node-gather-daemonset-6bq9d"
Jul 06 18:58:56.311678 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:56.311523    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/042ddd02-3a84-4075-8a71-bcbdd62b0108-podres\") pod \"perf-node-gather-daemonset-6bq9d\" (UID: \"042ddd02-3a84-4075-8a71-bcbdd62b0108\") " pod="openshift-must-gather-8xw5b/perf-node-gather-daemonset-6bq9d"
Jul 06 18:58:56.311678 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:56.311553    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/042ddd02-3a84-4075-8a71-bcbdd62b0108-sys\") pod \"perf-node-gather-daemonset-6bq9d\" (UID: \"042ddd02-3a84-4075-8a71-bcbdd62b0108\") " pod="openshift-must-gather-8xw5b/perf-node-gather-daemonset-6bq9d"
Jul 06 18:58:56.311678 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:56.311567    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/042ddd02-3a84-4075-8a71-bcbdd62b0108-proc\") pod \"perf-node-gather-daemonset-6bq9d\" (UID: \"042ddd02-3a84-4075-8a71-bcbdd62b0108\") " pod="openshift-must-gather-8xw5b/perf-node-gather-daemonset-6bq9d"
Jul 06 18:58:56.311678 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:56.311671    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/042ddd02-3a84-4075-8a71-bcbdd62b0108-sys\") pod \"perf-node-gather-daemonset-6bq9d\" (UID: \"042ddd02-3a84-4075-8a71-bcbdd62b0108\") " pod="openshift-must-gather-8xw5b/perf-node-gather-daemonset-6bq9d"
Jul 06 18:58:56.311907 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:56.311728    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/042ddd02-3a84-4075-8a71-bcbdd62b0108-podres\") pod \"perf-node-gather-daemonset-6bq9d\" (UID: \"042ddd02-3a84-4075-8a71-bcbdd62b0108\") " pod="openshift-must-gather-8xw5b/perf-node-gather-daemonset-6bq9d"
Jul 06 18:58:56.311907 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:56.311778    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/042ddd02-3a84-4075-8a71-bcbdd62b0108-lib-modules\") pod \"perf-node-gather-daemonset-6bq9d\" (UID: \"042ddd02-3a84-4075-8a71-bcbdd62b0108\") " pod="openshift-must-gather-8xw5b/perf-node-gather-daemonset-6bq9d"
Jul 06 18:58:56.323687 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:56.323653    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q5fk7\" (UniqueName: \"kubernetes.io/projected/042ddd02-3a84-4075-8a71-bcbdd62b0108-kube-api-access-q5fk7\") pod \"perf-node-gather-daemonset-6bq9d\" (UID: \"042ddd02-3a84-4075-8a71-bcbdd62b0108\") " pod="openshift-must-gather-8xw5b/perf-node-gather-daemonset-6bq9d"
Jul 06 18:58:56.429134 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:56.428955    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-8xw5b/perf-node-gather-daemonset-6bq9d"
Jul 06 18:58:56.633229 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:56.632817    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-8xw5b/perf-node-gather-daemonset-6bq9d"]
Jul 06 18:58:56.636030 ip-10-0-134-113 kubenswrapper[2580]: W0706 18:58:56.635970    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod042ddd02_3a84_4075_8a71_bcbdd62b0108.slice/crio-6cc76e751624dca8a8c09fdedd11686ca656742951a73874354d40cfbd294897 WatchSource:0}: Error finding container 6cc76e751624dca8a8c09fdedd11686ca656742951a73874354d40cfbd294897: Status 404 returned error can't find the container with id 6cc76e751624dca8a8c09fdedd11686ca656742951a73874354d40cfbd294897
Jul 06 18:58:56.951237 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:56.950257    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-8xw5b/perf-node-gather-daemonset-6bq9d" event={"ID":"042ddd02-3a84-4075-8a71-bcbdd62b0108","Type":"ContainerStarted","Data":"5e764bb80b621bfd7d240f4f3e501d8e22c4bbbada6df27a7a88307b37e0fde1"}
Jul 06 18:58:56.951237 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:56.950318    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-8xw5b/perf-node-gather-daemonset-6bq9d" event={"ID":"042ddd02-3a84-4075-8a71-bcbdd62b0108","Type":"ContainerStarted","Data":"6cc76e751624dca8a8c09fdedd11686ca656742951a73874354d40cfbd294897"}
Jul 06 18:58:56.951237 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:56.951115    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-8xw5b/perf-node-gather-daemonset-6bq9d"
Jul 06 18:58:56.977109 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:56.977045    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-8xw5b/perf-node-gather-daemonset-6bq9d" podStartSLOduration=0.977026126 podStartE2EDuration="977.026126ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 18:58:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:58:56.973800653 +0000 UTC m=+2472.131199141" watchObservedRunningTime="2026-07-06 18:58:56.977026126 +0000 UTC m=+2472.134424604"
Jul 06 18:58:59.275150 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:59.275122    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-lzzjn_24fc32df-2fce-41bf-8a8d-849b392bcb8f/dns/0.log"
Jul 06 18:58:59.299789 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:59.299765    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-lzzjn_24fc32df-2fce-41bf-8a8d-849b392bcb8f/kube-rbac-proxy/0.log"
Jul 06 18:58:59.358174 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:59.358142    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-x7rd5_d27a6ce1-0910-4c08-959a-550b8d468219/dns-node-resolver/0.log"
Jul 06 18:58:59.934837 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:58:59.934799    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-757fd97788-t78bl_128b0f11-7a7d-44eb-9830-5208fb456e64/registry/0.log"
Jul 06 18:59:00.003530 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:59:00.003501    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-kcr4l_c2ef9254-5ac3-45db-99a0-b5aff2a25d58/node-ca/0.log"
Jul 06 18:59:00.994276 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:59:00.994243    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4m2pkz_5d7cc344-17c6-4259-90cf-4422e436112b/istio-proxy/0.log"
Jul 06 18:59:01.186638 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:59:01.186599    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-68bf6db468-pvqn9_6f86a0d6-2cde-4e56-8348-9ad3351f9500/kube-auth-proxy/0.log"
Jul 06 18:59:01.220750 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:59:01.220720    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-tlmbg_75b7f81b-fdb8-4d09-8317-8c9323960ed5/istio-proxy/0.log"
Jul 06 18:59:01.255108 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:59:01.254991    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-tlmbg_75b7f81b-fdb8-4d09-8317-8c9323960ed5/istio-proxy/1.log"
Jul 06 18:59:01.293372 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:59:01.293344    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-59ldh_e46693cd-0f3e-4ed5-a5f5-748daad00f43/payload-pre-processing/0.log"
Jul 06 18:59:02.077730 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:59:02.077703    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-nmspz_534f2c50-cd67-4d03-90b9-5076223138ff/serve-healthcheck-canary/0.log"
Jul 06 18:59:02.580214 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:59:02.580184    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-89h8f_f354e957-d479-41ba-bf43-24303d500049/kube-rbac-proxy/0.log"
Jul 06 18:59:02.602621 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:59:02.602596    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-89h8f_f354e957-d479-41ba-bf43-24303d500049/exporter/0.log"
Jul 06 18:59:02.622936 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:59:02.622909    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-89h8f_f354e957-d479-41ba-bf43-24303d500049/extractor/0.log"
Jul 06 18:59:03.969772 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:59:03.969744    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-8xw5b/perf-node-gather-daemonset-6bq9d"
Jul 06 18:59:04.969323 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:59:04.969239    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-c977q_dfe46fe3-2f4d-4855-97c1-38e4e974c6f8/manager/0.log"
Jul 06 18:59:05.164442 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:59:05.164396    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-4vm88_e9cabd49-cdd7-4882-afc0-da156d82c239/manager/0.log"
Jul 06 18:59:05.210325 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:59:05.210297    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-79c74f6bbb-cbdff_8ba23aff-6404-4870-8919-6dc1ca89fa0c/maas-api/0.log"
Jul 06 18:59:05.238338 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:59:05.238257    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29722710-ldj78_44be2b8f-8aac-4423-aeea-03c7d2caea25/cleanup/0.log"
Jul 06 18:59:05.266931 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:59:05.266905    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29722725-dx9b7_17eeefb8-1ca4-4c82-b73f-fe031dbbd201/cleanup/0.log"
Jul 06 18:59:05.407494 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:59:05.407413    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-c84d79b86-pc4x6_ab01db64-3934-426a-a989-760248f8c0cd/manager/0.log"
Jul 06 18:59:05.430212 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:59:05.430185    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-rt527_a116c1ff-d114-4ba8-8a6d-eec4317c6978/server/0.log"
Jul 06 18:59:05.518804 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:59:05.518705    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-cjcm8_b77b21f6-57a3-476e-96c0-f9c87a61683e/manager/0.log"
Jul 06 18:59:05.692248 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:59:05.692215    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6bfdf69c66-tbm2m_6bc2ec3e-8ccd-41a1-a9d8-6013a0535dfb/manager/0.log"
Jul 06 18:59:05.715490 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:59:05.715461    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6bfdf69c66-tbm2m_6bc2ec3e-8ccd-41a1-a9d8-6013a0535dfb/copy-manifests/0.log"
Jul 06 18:59:05.742428 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:59:05.742344    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-69fb4594cf-859xw_c481500b-94ce-4b34-8a62-755f86931529/postgres/0.log"
Jul 06 18:59:07.250766 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:59:07.250734    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4m2pkz_5d7cc344-17c6-4259-90cf-4422e436112b/istio-proxy/0.log"
Jul 06 18:59:07.382486 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:59:07.382441    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-68bf6db468-pvqn9_6f86a0d6-2cde-4e56-8348-9ad3351f9500/kube-auth-proxy/0.log"
Jul 06 18:59:07.404856 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:59:07.404825    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-tlmbg_75b7f81b-fdb8-4d09-8317-8c9323960ed5/istio-proxy/0.log"
Jul 06 18:59:07.440888 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:59:07.440846    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-tlmbg_75b7f81b-fdb8-4d09-8317-8c9323960ed5/istio-proxy/1.log"
Jul 06 18:59:07.469381 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:59:07.469340    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-59ldh_e46693cd-0f3e-4ed5-a5f5-748daad00f43/payload-pre-processing/0.log"
Jul 06 18:59:08.725545 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:59:08.725518    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4m2pkz_5d7cc344-17c6-4259-90cf-4422e436112b/istio-proxy/0.log"
Jul 06 18:59:08.856550 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:59:08.856505    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-68bf6db468-pvqn9_6f86a0d6-2cde-4e56-8348-9ad3351f9500/kube-auth-proxy/0.log"
Jul 06 18:59:08.882857 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:59:08.882827    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-tlmbg_75b7f81b-fdb8-4d09-8317-8c9323960ed5/istio-proxy/0.log"
Jul 06 18:59:08.919524 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:59:08.919494    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-tlmbg_75b7f81b-fdb8-4d09-8317-8c9323960ed5/istio-proxy/1.log"
Jul 06 18:59:08.949348 ip-10-0-134-113 kubenswrapper[2580]: I0706 18:59:08.949312    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-59ldh_e46693cd-0f3e-4ed5-a5f5-748daad00f43/payload-pre-processing/0.log"